We need to go deeper…

[FrenchBen]

Thank you for detailing your CbyGE adventures, and providing a full package for it.
Now this replicates the commands that the app would send to the cbyge server, but I would be curious if it wouldn't be faster to skip the server altogether.

Would you be willing to help a not so network inclined individual capture the packets sent and received by the actual device, to completely bypass the need to talk to the socket or GE web endpoints?

Unless I missed it, this would allow you to talk directly to the device within your network, without it going all the way out to the CbyGE mothership?

The backstory is that I have a now unsupported device that I'd like to keep using :)

[unixpickle]

I haven't tried looking at the protocol the bulbs use, but my guess is it will be similar to the one the app uses since programmers are usually lazy. Another option might be to dig into the Bluetooth protocol instead of using TCP. I saw something like that here.

For going the TCP route: if you are able to capture the packets, you'll likely still need a hackable router to MiTM the bulbs. The more annoying case would be if the bulbs have the IP of the server hardcoded. If not, you could find the domain name it uses and setup a local DNS server on your network to point that domain name to your own IP. This would still require changing a setting on the router the cbyge device is connected to.

Maybe it's even possible to overwrite the firmware on the bulbs via a fake software update or something, akin to jailbreaking, but that seems like a much harder endeavor.

[FrenchBen]

Capturing the packets has been quite difficult, and with the device no longer supported, I can't even see the incoming requests for it.
Thanks for sharing the google code, it will be handy to go through. CbyGE runs a busybox env, with a few binaries that seem to run in a loop.
As I'm getting more familiar with exploring the hardware/firmware, I was hoping that an existing device could provide some insights on the TCP requests sent, or how to explore them.

Firmware upgrade is a possible future, I need to get a solid start of how it all works first. :)

[micromashor]

Hi! Not sure if either of you are still interested in this, but I'll share my findings and methods so far:

I'll share my full procedure down below, but if you just want a summary of what I've found, here it is:

1. The bulbs perform an unauthenticated standard DNS query for cm-ge.xlink.cn. This can be easily altered by way of a local DNS server if you wish to MITM its traffic.
2. The protocol it uses is wrapped in TLS, although the bulbs do not validate the server's certificate, so any self-signed certificate (easily generated with OpenSSL) will do.
3. The bulbs communicate with the server over TCP on port 23779.
4. The protocol used by the bulbs is an unfamiliar binary protocol, but as already mentioned, programmers are lazy, and it's probably a different proprietary protocol being reused.

That's a basic summary of my findings, but if you want the full story, read on:

1. I set up a raspberry pi running HostAPd, the software to make the pi operate as a WiFi Access Point (AP), and set up appropriate adapter bridging, DHCP server, and (this will become significant later) a local DNS server, so that the bulbs would be able to communicate with their server, but not without passing their traffic under my watchful eye.
2. I then factory-reset a bulb and had it connect to my pi AP.
3. By running Wireshark on the pi, I was able to see that as soon as the bulbs are powered on, they send a DNS query for the name cm-ge.xlink.cn, which I see was also hard-coded in the app. They then establish a standard TCP connection to the returned IP address on port 23779.
4. In order to intercept the traffic, I added a local DNS entry on the pi, causing it to serve up the LAN-side address of my PC, rather than the cloud server's address.
5. I spun up a "packet proxy" similar to what's documented in the readme (although with NodeJS since that's my most comfortable language), and started sniffing the traffic with Wireshark. Immediately noticed that all of its traffic is encrypted with TLSv1.2. Bummer.
6. For no reason other than pure impulsiveness, I ran a TLS connection test to cm-ge.xlink.cn, and immediately saw an alert that the certificate was 10 months past expiration. This generally means that the TLS implementation being used by the bulbs is lazy, or bare-bones, and is not performing proper date validation on the certificate, or any for that matter.
7. To test my theory, I used openssl to whip up a self-signed certificate for cm-ge.xlink.cn, upgraded my NodeJS packet proxy to become TLS-aware, and installed this certificate. The bulb happily accepted this forged certificate, and proceeded with its connection to the cloud, allowing me to successfully MITM all of its traffic, giving me access to the plaintext stream. I modified my NodeJS packet proxy so that it would log all of the traffic passed across the connection, and save it to the disk.
8. I was somewhat disappointed when I checked the log file, as it was an unfamiliar binary protocol (rather than the easily-recognized HTTP or WebSocket protocols), so I started googling (in hindsight, I probably should have started with that) and found this repo.

@FrenchBen - Since you asked about communicating within the LAN, without needing to go out to the CbyGE mothership, that seems totally possible, although you'll probably have to host a server on your network. If we could find a way to replace the firmware on the bulbs, it'd be easy enough to expose a UDP listener or some other sort of direct-control API, which would eliminate the need for a server.

Hope this info helps!

PS if anyone wants to generate a self-signed certificate for tinkering purposes, this will do it (provided openssl is installed):
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -subj '/CN=cm-ge.xlink.cn' -sha256 -days 3650 -nodes

[EAuckland]

I'm very interested in getting this to work with home assistant however what you guys are doing is way over my head lol. Please let me know if I can help at all.

…

On Mon, Jan 3, 2022, 18:50 micromashor ***@***.***> wrote: Hi! Not sure if either of you are still interested in this, but I'll share my findings and methods so far: I'll share my full procedure down below, but if you just want a summary of what I've found, here it is: 1. The bulbs perform an unauthenticated standard DNS query for cm-ge.xlink.cn. This can be easily altered by way of a local DNS server if you wish to MITM its traffic. 2. The protocol it uses is wrapped in TLS, although the bulbs do not validate the server's certificate, so any self-signed certificate (easily generated with OpenSSL) will do. 3. The bulbs communicate with the server over TCP on port 23779. 4. The protocol used by the bulbs is an unfamiliar binary protocol, but as already mentioned, programmers are lazy, and it's probably a different proprietary protocol being reused. That's a basic summary of my findings, but if you want a full summary of my findings, read on: 1. I set up a raspberry pi running HostAPd, the software to make the pi operate as a WiFi Access Point (AP), and set up appropriate adapter bridging, DHCP server, and (this will become significant later) a local DNS server, so that the bulbs would be able to communicate with their server, but not without passing their traffic under my watchful eye. 2. I then factory-reset a bulb and had it connect to my pi AP. 3. By running Wireshark on the pi, I was able to see that as soon as the bulbs are powered on, they send a DNS query for the name cm-ge.xlink.cn, which I see was also hard-coded in the app. They then establish a standard TCP connection to the returned IP address on port 23779. 4. In order to intercept the traffic, I added a local DNS entry on the pi, causing it to serve up the LAN-side address of my PC, rather than the cloud server's address. 5. I spun up a "packet proxy" similar to what's documented in the readme (although with NodeJS since that's my most comfortable language), and started sniffing the traffic with Wireshark. Immediately noticed that all of its traffic is encrypted with TLSv1.2. Bummer. 6. For no reason other than pure impulsiveness, I ran a TLS connection test to cm-ge.xlink.cn, and immediately saw an alert that the certificate was 10 months past expiration. This generally means that the TLS implementation being used by the bulbs is lazy, or bare-bones, and is not performing proper date validation on the certificate, or any for that matter. 7. To test my theory, I used openssl to whip up a self-signed certificate for cm-ge.xlink.cn, upgraded my NodeJS packet proxy to become TLS-aware, and installed this certificate. The bulb happily accepted this forged certificate, and proceeded with its connection to the cloud, allowing me to successfully MITM all of its traffic, giving me access to the plaintext stream. I modified my NodeJS packet proxy so that it would log all of the traffic passed across the connection, and save it to the disk. 8. I was somewhat disappointed when I checked the log file, as it was an unfamiliar binary protocol (rather than the easily-recognized HTTP or WebSocket protocols), so I started googling (in hindsight, I probably should have started with that) and found this repo. @FrenchBen <https://github.com/FrenchBen> - Since you asked about communicating within the LAN, without needing to go out to the CbyGE mothership, that seems totally possible, although you'll probably have to host a server on your network. If we could find a way to replace the firmware on the bulbs, it'd be easy enough to expose a UDP listener or some other sort of direct-control API, which would eliminate the need for a server. Hope this info helps! PS if anyone wants to generate a self-signed certificate for tinkering purposes, this will do it (provided openssl is installed): openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -subj '/CN=cm-ge.xlink.cn' -sha256 -days 3650 -nodes — Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWDFOE3RQDXY5VR2AYQ5CHTUUJG7NANCNFSM5FBPLIVQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Rattkener]

• on this. I'm going to be digging into reversing the binary protocol, I think with out any expertise on the topic my next step would be to make changes via the app and diff those changes to try and detect patterns.. Very brute force but that's all I got

[micromashor]

That's been my process as well, but you'll very quickly notice that there's an exchange of 4-5 messages back and forth every time you make a change in the app. I'm going to work on a way to be able to selectively replay these messages to see what they change, but it's quite difficult to figure out.

Also be warned that you should remove the power to all devices except one, as I've noticed that when multiple devices are on at the same time, they start RSTing connections very unpredictably, or make sure your packet proxy is ready to gracefully handle any socket being closed by the remote host with RST. Not really sure what's going on there.

[mrja]

I'll leave this here... I don't have time (read: the skill to do so in a timely manner) to figure this out but after a couple hundred hours bashing my head against the wall without a debugger, I'm motivated to help:

http://wiki.telink-semi.cn/wiki/protocols/Telink-Mesh/

namely, the SDK.

telink_mesh_sdk.zip

if this doesn't contain 99% of what drives the C by GE firmware, I'll eat my own shorts.

[CaptainN]

I've been looking to try and integrate these lights with hass for a bit. I have a couple of notes, in no particular order. I'd love to help write a hass integration for the cloud, and possibly a separate version, if we can solve some of the BLE mesh issues with the "direct connect" bulbs.

I have gotten this project (cbyge) to work perfectly, without any trouble. It just magically works - very cool. I think the information in this code would serve as a basis for a cloud integration in hass. I'm not sure if the golang code can be used directly or it would require a port to Python. I haven't gotten that far in the hass docs yet.

The short of what I've found for controlling these lights locally is that while there is a way to query for the info needed to access the BLE mesh network, I can't seem to gain control using any of the existing tools. There are 2 different integrations that partially work, and both are based on the techniques describe in python-laurel:

• There is a hass gelight integration, written in Python that doesn't really work well. The data fetch is kind of broken, but with a bit of effort the config can be set to something that I think is correct. But it won't connect to the mesh. It just times out trying. The computer hass is hosted on does seem to pair and connect to one of the bulbs reliably, but hass can't seem to see it.
• There is also cync2mqtt, which is more cleanly written, but has a similar issue connecting to the mesh. That doesn't mean it can't be made to work, but I haven't had any luck so far.

Both of these solutions have tools to produce configuration files that grab too much data, and cause issues if not edited down (they seem to dump "group" data, which is unsupported by either tool).

For connecting to the mesh network using BLE, I've wondered if the older style bulbs (non-direct connect) will mesh with the newer direct connect bulbs, and if that would provide a way in. From what I understand, we only need hass to connect to one device in the mesh, to be able to control the rest. I don't have any old style bulbs to test that with, but I do wonder whether it would work.

Anyway, it would be great to get a hass cloud integration together, if nothing else. A mitm solution might also be kind of neat, but any basic control would be welcome, even if it uses the cloud.

I may take a stab at it, if I can find some time. Does anyone know of a good base to start from - some example cloud integration template in hass or similar?

[micromashor]

I'm not entirely sure how hass works, but from what I've found, MITMing the network traffic seems to be the least finicky of the approaches. I tried tinkering with BLE/mesh and it was extremely unstable.

Unfortunately, we don't really have a great base to start from, since we have no documentation on how the network protocol operates. I've tried reading through the code in this repository to see if I can reverse-engineer it and figure out how the network protocols work, but it's just ended in ripped-out hairs because I am not familiar enough with Go to make any sense of it.

If you'd like, I can document my approach for MITMing the network traffic to the bulbs and the little bit of code that accompanies it. It requires a Raspberry Pi wired in via Ethernet.

We really need to figure out how this network protocol works (either by reverse-engineering or from the repository owner) in order to go any further with developing a solution that interfaces directly with the bulbs, bypassing the CbyGE cloud.

[CaptainN]

It seems like it should be possible to do something with BLE mesh - since that's literally how the battery power switch I have works, and that device does work reliably. Hmm, I wonder if having that switch in my mesh is causing some of the troubles with connecting?

I also though it might be easiest to extend cbyge to support mqtt messaging. In that case, we could just run this server, and use an existing mqtt integration in hass. That might be even easier than porting it to python and creating a new cloud integration. It would still require understanding golang (though at least it's not Python!)

[CaptainN]

It looks like it should be relatively easy to just hit the rest API on a cbyge instance, directly from a hass light integration. I don't even think this will require changing any existing code!

[FrenchBen]

Some details on the chipset for my use case - it's a Linkplay A31 v04, I tried to email the company to see if they would be willing to share how I can reflash/update, but no dice so far. Specs are here: https://www.linkplay.com/modules-wi-fi-2

There's been a little bit of work here from @Crymeiriver : https://github.com/Crymeiriver/LS150

[CaptainN]

I wrote a simple bridge between this project and hass through MQTT. Check it out!

https://github.com/CaptainN/cbyge2mqtt

[Rattkener]

This project works well enough as a client to the Cync service, it's removing the sketchy api that I'm most worried about.
BLE access is very strictly limited in the newer bulbs as when they changed from C by GE to Cync I believe they sold it to a 3rd party but continued to sell under their name. I couldn't see GE natively trying to use a Chinese API service.

Politics aside, they force network access (and the app repeatedly demands location information despite not needing it)and have discontinued almost all BLE add in devices and hubs. They don't want any way for you to not use the app.

My work to cut the cloud out has been met with pretty much failure up to this point. I was able to shut them off and turn them on but that's the end of it. Hoping that reviewing the updated info will let me come back with better info, be back soon :)

[CaptainN]

From what I've gathered, these bulbs use a proprietary Telink mesh (I used nRF Connect app on Android to figure that out). They do have devices sold under the Cync brand like battery powered light switches. These have no wifi capability. I'd imagine they'll want to support those for at least a while, and they do work reliably.Is there any info on telinks website that could help? http://wiki.telink-semi.cn/wiki/protocols/Telink-Mesh/

[micromashor]

@Rattkener That's exactly my reason for being here. I don't trust a nameless API service (especially one that doesn't validate that it's actually connecting to the API service), but also for my purposes, I despise the sometimes hundreds of milliseconds latency that comes with using that external service. I run almost all of my other devices through a server hosted locally on my network, so I can achieve response times as low as a few ms.

As of today, this API service is no longer nameless, as their website is back up. I'll try reading through their developer docs and see if I can find anything re: Binary API Protocol, and update here if I find anything.

The issue, as you mentioned, is figuring out how the entirely undocumented API works. I've asked the repository owner to share documentation regarding the app-to-API protocol, as they appear to be very similar (if not the same), but I have yet to find such documentation.

[CaptainN]

I integrated with home Assistant through MQTT directly in cbyge . The result is FAR more stable than the previous attempt through node/REST. It still is using the cloud integration, but it works!

[sporkman]

At any point does the app ever talk directly to the bulbs over wifi?

Any ideas on why setting up "scenes" in the app only works over bluetooth?

I just bought a few more of these last week, newegg had RGB light strips on sale. I also have 3 outdoor floods. If anyone wants me to gather info on these particular devices, let me know. I hate that we have to talk to some dodgy cloud server for all this, the idea that you lose control of your lights if your internet goes away is nuts.

[CaptainN]

They actually run on a BT mesh network, which is pretty cool. It looks like it should be possible to jump right in to that mesh too, and control things as if we were as switch (thats how their battery powered switches work). I can inspect that mesh, get the mac addresses, and even connect to it from my phone, and HA server.

But I gave up. I use these only in the basement, with a BT Switch, and removed them from the rest of my house.

I have been replacing them with some Zigbee Ledvance bulbs. Those are inexpensive, have freely available firmware, and work great with Conbee II. No need to jump through hoops with GE bulbs. If they ever open their API, they'd be great devices, but until then, they are only mildly useful.

[reidab]

I haven't had time to dig into these further, but there seems to be a decent amount of Chinese-language documentation and examples in the https://github.com/xlink-corp github org and on https://docs.xlink.cn/

This doc appears to describe the details of the serial protocol used by the devices: https://docs.xlink.cn/download/attachments/1703987/WIFI%E9%80%8F%E4%BC%A0%E4%B8%B2%E5%8F%A3%E9%80%9A%E8%AE%AF%E5%8D%8F%E8%AE%AEV2.52.docx?version=1&modificationDate=1513584293000&api=v2

[iburistu]

I've been independently working on a LAN controller for Cync devices and just happened upon this thread. PoC code that I've put together can be found here. It's a bit rough around the edges but I've been using it for the past few months without too many issues to control lightbulbs and smart plugs. No idea what the actual APIs are for the devices but I was able to figure out the commands I cared about by watching the TCP messages and through trial and error.

I've been unsuccessful with interfacing with a RGB lightstrip...I can usually get it to work for a few days, but it wises up to the MITM somehow (BT from phone?) and refuses to work unless I factory reset.

@sporkman are your lights the "direct connect" version or BT? They might be compatible with this code, but I'm only rocking older model lights & plugs. New firmware might be more intelligent?

[sporkman]

@sporkman are your lights the "direct connect" version or BT? They might be compatible with this code, but I'm only rocking older model lights & plugs. New firmware might be more intelligent?

The light strips are BT-only, the floods are BT and wifi if that clarifies things?