Protocol notes

[uzlonewolf]

I wanted to collect my notes about the new 6699/v3.5 protocol in one place without burying it in back-and-forth troubleshooting discussion, so here it is.

---

Packet formats

As a recap, the older 55AA/v3.1-v3.4 packet format looks like:

000055aaSSSSSSSSMMMMMMMMLLLLLLLL[RRRRRRRR]DD..DDCC..CC0000aa55 where:
000055aa - prefix
SSSSSSSS - 32-bit sequence number
MMMMMMMM - 32-bit Command ID
LLLLLLLL - 32-bit packet length - count every byte from the return code through (and including) the footer
[RRRRRRRR] - packets from devices have a 32-bit return code. Packets from the app/client do not have this field
DD..DD - variable length encrypted payload data
CC..CC - checksum, either 32-bit (4-byte) CRC32 for v3.1-v3.3, or 256-bit (32-byte) HMAC-SHA256 for v3.4
0000aa55 - footer

Everything except the payload data is sent unencrypted/in the clear. Payload data is padded to have a multiple-of-16 length and is AES128 encrypted in ECB mode. All multi-byte integers are big-endian.

The newer v3.5 format looks like:

00006699UUUUSSSSSSSSMMMMMMMMLLLLLLLL(II*12)DD..DD(TT*16)00009966
00006699 - prefix
UUUU - Unknown 16-bit or 2*8-bit field. Current devices (as of June 2024) always transmit as 0x0000 and completely ignore on reception
SSSSSSSS - 32-bit sequence number
MMMMMMMM - 32-bit Command ID
LLLLLLLL - 32-bit packet length - count every byte from the IV/nonce through (and including) the Tag, but not the footer
(II*12) - 96-bit (12-byte) per-packet AES-GCM IV/nonce
DD..DD - variable length encrypted payload data
(TT*16) - 128-bit (16-byte) signature Tag value from AES-GCM encrypt-and-sign
00009966 - footer

Everything except the payload data is sent unencrypted/in the clear. Payload data is not padded and is AES128 encrypted in GCM mode. The GCM Tag value is computed while encrypting/decrypting; the UUUU through LLLLLLLL fields are signed but not encrypted while the payload is both signed and encrypted. The GCM IV/nonce changes for every packet and is in the II*12 field.

Packets from devices still have a 32-bit return code, however it is encrypted and prepended to the payload and needs to be stripped after decryption but before payload processing.

---

Session keys

Both v3.4 and v3.5 use a 3-way session key negotiation scheme to generate unique per-connection session keys to encrypt the payload data. All payloads during this exchange are encrypted like normal using the real key.

First, the client sends the device a 16-byte nonce. Next, the device responds with its own 16-byte nonce plus a 32-byte HMAC-SHA256 of the client nonce (again using the real key as the key). Finally, the client sends the device the 32-byte HMAC-SHA256 of the device nonce.

The session key is then calculated by XORing the nonces and encrypting the result. The encryption routine used for this is the exact same as for payloads; the v3.5 GCM IV is the first 12 bytes of the client nonce. I.e.:

tmp_key = bytes( [ a^b for (a,b) in zip(device_nonce, client_nonce) ] )
cipher = tinytuya.AESCipher( real_key )
session_key = cipher.encrypt( tmp_key, use_base64=False, pad=False, iv=client_nonce[:12] )[12:28]

---

Device discovery broadcast packets

v3.5 devices also use the new 6699 packet format for their discovery broadcasts. For these, the encryption key is the MD5 digest of yGAdlopoPVldABfn. AES-GCM decryption and Tag verification is then performed as usual. A small stand-alone script to receive and decrypt these broadcasts is:

from Crypto.Cipher import AES
from hashlib import md5
import socket

clients = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
clients.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
clients.bind(("", 6667))
clients.settimeout(7)

key = md5( b'yGAdlopoPVldABfn' ).digest()

for i in range( 10 ):
    data, addr = clients.recvfrom(4048)
    prefix = data[:4]

    if prefix == b'\x00\x00\x55\xAA':
        print( 'Ignoring 55AA packet' )
    elif prefix == b'\x00\x00\x66\x99':
        header = data[4:18]
        iv = data[18:30]
        payload = data[30:-20]
        tag = data[-20:-4]

        print( addr, '=', payload )

        cipher = AES.new(key, AES.MODE_GCM, nonce=iv)
        cipher.update( header ) # needed for tag verification
        decrypted = cipher.decrypt_and_verify( payload, tag )

        print( 'Decrypted:', decrypted )
        print( 'Return Code:', decrypted[3]) # really decrypted[:3]
        print( 'Payload:', decrypted[4:])
    else:
        print( 'Unknown packet format!', prefix )

    print('')

Once decrypted these new devices are reporting protocol v3.5.

---

Device discovery packets redux

Newer v3.5 devices do not send out unsolicited discovery broadcasts. Instead, they listen for broadcasts from a client app and send their discovery packet directly to that client.

The client broadcast contains the payload {"from":"app","ip":"192.168.1.42"} and is sent to the subnet broadcast address (i.e. 192.168.1.255) on port 7000. It is GCM encrypted the same way as broadcasts above.

When a device receives a client broadcast, it responds by sending a device discovery packet directly to the IP address specified in the client broadcast. This again uses port 7000 and is GCM encrypted the same way as broadcasts above. The device I used in my testing will send the device discovery packet out as a broadcast if the client app specifies "255.255.255.255" as the IP address, however I suspect this is an accident and is not something they intended to be used.

---

"Version Protocol" header

The 3.xCCCCCCCCSSSSSSSSUUUUUUUU header found in v3.3+ commands breaks down as follows:

3.x - The protocol version as a string, i.e. "3.3"
CCCCCCCC - When the message comes from the MQTT server, this is a crc32 hash of the message; when it comes from the LAN it is set to 0.
UUUUUUUU - Unique source identifier. The device is source 1 and clients should generate a unique (random?) number to identify that specific client - this identifier should persist across connections.
SSSSSSSS - Source-specific message sequence number that increments on every message and also persists across connections. If a device receives a 2nd message from the same source with the same or decreasing sequence number, that message is discarded. This allows a client to send a message via both the cloud and the local network simultaneously and the device will process whichever arrives first and ignore the other. The client can also use it to figure out if it has missed any messages by looking for jumps in the sequence number.

If the unique source identifier is "0" then the sequence number is ignored and all messages are processes.

The device I looked at only "remembers" the last 5 unique source identifiers; the oldest is overwritten when a new source shows up.

[uzlonewolf]

Added "Device discovery broadcast packets" section.

[uzlonewolf]

Reworked discovery example to be a bit more complete.

[Ahmad-Agbaria]

shouldn't you change clients.bind(("", 6667)) to clients.bind(("", 6699)) as you mentioned in notes that v3.5 uses this port

[uzlonewolf]

No, 00006699 is the header prefix (replacing 000055AA), it still uses ports 6667/6668.

[uzlonewolf]

Added:

"Version Protocol" header

The 3.xCCCCCCCCSSSSSSSSUUUUUUUU header found in v3.3+ commands breaks down as follows:

3.x - The protocol version as a string, i.e. "3.3"
CCCCCCCC - When the message comes from the MQTT server, this is a crc32 hash of the message; when it comes from the LAN it is set to 0.
UUUUUUUU - Unique source identifier. The device is source 1 and clients should generate a unique (random?) number to identify that specific client - this identifier should persist across connections.
SSSSSSSS - Source-specific message sequence number that increments on every message and also persists across connections. If a device receives a 2nd message from the same source with the same or decreasing sequence number, that message is discarded. This allows a client to send a message via both the cloud and the local network simultaneously and the device will process whichever arrives first and ignore the other. The client can also use it to figure out if it has missed any messages by looking for jumps in the sequence number.

If the unique source identifier is "0" then the sequence number is ignored and all messages are processes.

The device I looked at only "remembers" the last 5 unique source identifiers; the oldest is overwritten when a new source shows up.

[uzlonewolf]

Added:

Device discovery packets redux

Newer v3.5 devices do not send out unsolicited discovery broadcasts. Instead, they listen for broadcasts from a client app and send their discovery packet directly to that client.

The client broadcast contains the payload {"from":"app","ip":"192.168.1.42"} and is sent to the subnet broadcast address (i.e. 192.168.1.255) on port 7000. It is GCM encrypted the same way as broadcasts above.

When a device receives a client broadcast, it responds by sending a device discovery packet directly to the IP address specified in the client broadcast. This again uses port 7000 and is GCM encrypted the same way as broadcasts above. The device I used in my testing will send the device discovery packet out as a broadcast if the client app specifies "255.255.255.255" as the IP address, however I suspect this is an accident and is not something they intended to be used.

[jasonacox]

Great discovery, @uzlonewolf ! Is this exclusive to 3.5 or do other versions also see this 7000 broadcast and respond back?

In any case, it would be good to add this to our scanner.

[uzlonewolf]

The broadcast from the app uses GCM and the 6699 packet format, so I suspect we'll only see it in v3.5 devices. In theory they could add a 55AA packet as well, but I find that unlikely as it would require a firmware update to older devices and that's not something they really do (I think I've seen maybe 3 devices get firmware updates).

[uzlonewolf]

And yes, I do intend to add it to the scanner as well as find_device() (which is used by address='Auto'; I'll probably just rewrite this to use the scanner so we don't have to implement this in multiple places). As I mentioned in 507 the tricky part is going to be getting the list of local IP addresses/interfaces in python on multi-interface machines.