Use with device already in the network?

[YeapGuy]

Hi,
How technically feasible is it to use this project to work with official AirTags or other Find My devices? Already working AirTag clones are being sold for $2-4 a piece on Aliexpress, so I don't see a point in spending a lot of time messing with flashing, firmwares and all of that OpenHaystack stuff, when I can just buy a working "AirTag" for so cheap.

[malmeloo]

The issue with using original (or compatible) accessories is that each accessory has a private master key that is required in order to decrypt its location reports. This key is generated during the pairing session between the accessory and an Apple device when it is first set up.

The nicest solution would be to perform the pairing sequence ourselves in order to obtain the private key. We have pretty good (official) documentation for the bluetooth protocol between the device and accessory, so that won't be a problem. The real issue is that Apple's servers are also involved in the pairing process. As far as I know, nobody has successfully reverse engineered this part of the protocol yet, so we would first need to find out what data is exchanged between the device and Apple. Well-behaving accessories will reject pairing attempts that are not signed by Apple servers.

Another way could be to have an Apple device do the pairing, and figure out where the master key is stored. As long as we have enough details to generate the rotating private keys derived from this master key, we can fetch location reports for the accessory. I don't own any Apple devices however, so I'm not really able to help in this regard.

That said, this is something I'd be interested in to include in this library, so I'll leave this issue open as a feature request.

[YeapGuy]

I successfully retrieved the keys from the macOS Find My application.

What's next?

[malmeloo]

Oh, neat! I'm expecting the privateKey value to be the master key from which we can derive the accessory's keys and retrieve its location reports. I don't have much time to replicate the exact algorithm right now, but I'll look into it by the end of the week. Feel free to poke me if I forget :)

Just for future reference, can you share where you found those values? This is information I'd like to include in the documentation if possible.

[malmeloo]

Just as a quick update, I'm currently working on implementing the key derivation algorithm needed for this. It's just taking a little longer because to my knowledge this algorithm isn't standardized anywhere, so I have to roll my own crypto... fingers crossed, haha.

At this point, judging by your screenshot I'm fairly certain that we're going to need the secondarySharedSecret in combination with either the privateKey or the publicKey (they appear to be the same in your pic? the first few bytes at least.) The sharedSecret is probably the one used to derive the "primary key," which is only used in lost mode on the first day until 4 am. I think the "secondary key" is more interesting, but both use the same algorithm, so it's not difficult to implement both.

In the meantime, could you please try to dump a copy of your tag in lost mode using this script? There is a keyroll mechanism involved, so to test the algorithm we're gonna sequentially generate a bunch of keys and compare them against one of the public keys that the tag broadcasts; if it's in the sequence, we know the algorithm works. If you have a rough indication of when you first paired your tag, that will help narrow down the search sequence. The keyroll works in periods of 15 minutes.

[YeapGuy]

Just for future reference, can you share where you found those values? This is information I'd like to include in the documentation if possible.

Sure :) There are .record files in /Library/com.apple.icloud.searchpartyd/OwnedBeacons/ ‒ one for each "owned beacon". These files are actually plists. The data in them is ‒ of course ‒ encrypted. It's using AES GCM. The first value of the plist is the nonce, the second value is the tag and the third value is the data.
The key used to encrypt these .record files can be obtained by running security find-generic-password -l 'FindMyAccessories' -g ‒ the "gena" attribute value is the key in hex format.
Here's the script I used to decrypt the .record file. It's very rough and you have to substitute the key and the .record file path, but it works.

in combination with either the privateKey or the publicKey (they appear to be the same in your pic? the first few bytes at least.)

The privateKey includes the publicKey at the beginning, but they're not the same ‒ the privateKey is longer.

If you have a rough indication of when you first paired your tag, that will help narrow down the search sequence. The keyroll works in periods of 15 minutes.

Sure, I'll re-pair the tag when I'm at home, so that I know the exact time when it was paired, and then I'll give it a shot.

[YeapGuy]

Okay, I re-paired the tag at 13:28, put it into lost mode at 13:30 (why is this needed btw?) and turned off my experimental iPhone right after that. Then I ran the scanner.

Here's the result from 13:32 (4 minutes after pairing)

Device - C8:EC:6B:63:BA:BC
  Public key:   COxrY7q8bBtoDHzbAgL282zjCvC516HZGV0jgg==
  Lookup key:   0WN0SQ78QJ5F/DVz5Wy0OJgstsebL07Av8iIPLpUDu0=
  Status byte:  20
  Hint byte:    c7
  Extra data:
    Adapter             : /org/bluez/hci0
    Address             : C8:EC:6B:63:BA:BC
    AddressType         : random
    Alias               : C8-EC-6B-63-BA-BC
    Blocked             : False
    Connected           : False
    LegacyPairing       : False
    ManufacturerData    : {76: bytearray(b'\x12\x19 l\x1bh\x0c|\xdb\x02\x02\xf6\xf3l\xe3\n\xf0\xb9\xd7\xa1\xd9\x19]#\x82\x00\xc7')}
    Paired              : False
    RSSI                : -49
    ServicesResolved    : False
    Trusted             : False
    UUIDs               : []

And here's the result from 13:55 (27 minutes after pairing)

Device - C8:EC:6B:63:BA:BC
  Public key:   COxrY7q8bBtoDHzbAgL282zjCvC516HZGV0jgg==
  Lookup key:   0WN0SQ78QJ5F/DVz5Wy0OJgstsebL07Av8iIPLpUDu0=
  Status byte:  20
  Hint byte:    36
  Extra data:
    Adapter             : /org/bluez/hci0
    Address             : C8:EC:6B:63:BA:BC
    AddressType         : random
    Alias               : C8-EC-6B-63-BA-BC
    Blocked             : False
    Connected           : False
    LegacyPairing       : False
    ManufacturerData    : {76: bytearray(b'\x12\x19 l\x1bh\x0c|\xdb\x02\x02\xf6\xf3l\xe3\n\xf0\xb9\xd7\xa1\xd9\x19]#\x82\x006')}
    Paired              : False
    RSSI                : -49
    ServicesResolved    : False
    Trusted             : False
    UUIDs               : []

Do you need anything else? I'm willing to send you over the stuff I dumped from the macOS Find My app if that helps. 😄

[malmeloo]

Thanks! With lost mode I actually meant the "separated" state, i.e. the state in which the tag is actually broadcasting its keys and can be found using Find My. Those scans are looking good though! It keeps broadcasting the same key while the hint byte changes, which is what I was expecting.

Yesterday I implemented the actual algorithm, so it should be able to generate the keys now. I did have to change it a bit to generate private keys instead of public ones (because we want to decrypt reports instead of only generate them), so I hope it works... my group theory is a bit rusty. 😅 If I find the time, I'll try to integrate it with the library today and see if I can set up a test script.

If you're comfortable with sharing that data, it would be greatly appreciated! Feel free to send them to git@mikealmel.ooo.

[airy10]

The key used to encrypt these .record files can be obtained by running security find-generic-password -l 'FindMyAccessories' -g ‒ the "gena" attribute value is the key in hex format.

Which OS are you running ? I don't have that record in my Keychain
But I'm running 14.4 beta and Apple did some changes about Localisation data on that system (for example, plists in Library/Caches/com.apple.findmy.fmipcore used to contain plain data but is now encrypted data).

[YeapGuy]

I'm running 13.6.1 on a Hackintosh system

[malmeloo]

Alright, it should be ready! I have added a new example which you can check out if you want; it currently does not actually fetch location reports, just tries to find the private key belonging to the broadcasted public key. That key can be plugged into the fetch_reports.py example though.

Note that the master / secret keys are in binary format, so you might have to convert them first. I think the script should be pretty self-explanatory, but if you get stuck let me know.

[YeapGuy]

The key used to encrypt these .record files can be obtained by running security find-generic-password -l 'FindMyAccessories' -g ‒ the "gena" attribute value is the key in hex format.

Which OS are you running ? I don't have that record in my Keychain But I'm running 14.4 beta and Apple did some changes about Localisation data on that system (for example, plists in Library/Caches/com.apple.findmy.fmipcore used to contain plain data but is now encrypted data).

Heh, funny, the command stopped working for me as well. The keychain item is now called BeaconStore, so the command to obtain the key is now security find-generic-password -l 'BeaconStore' -g. I didn't do any software update ‒ Apple is somehow making changes on the fly. Funny, but annoying. Hopefully they don't do more significant changes.

[airy10]

Ah right.
The 'gena' attribute is null for me but the password works for me

func cmd(_ cmd: String, _ args: String...) -> String {
    let task = Process()
    task.launchPath = cmd
    task.arguments = args
    let pipe = Pipe()
    task.standardOutput = pipe
    task.launch()
    task.waitUntilExit()
    let data = pipe.fileHandleForReading.readDataToEndOfFile()
    guard let output: String = String(data: data, encoding: .utf8) else { return "" }; 
    return output
}

let hexKey = cmd("/usr/bin/security", "find-generic-password",  "-l",  "BeaconStore",  "-w")

[airy10]

I've updated your script to automatically get the decode key and decode all of the files from ~/Library/com.apple.icloud.searchpartyd
https://gist.github.com/airy10/5205dc851fbd0715fcd7a5cdde25e7c8

[malmeloo]

It works! I was successfully able to retrieve the private key belonging to the scan @YeapGuy posted here before 🎉 The one you emailed me earlier today does not work however; I'm not entirely sure why, but I probably messed something up for the secondary key generation. After the first 4:00 am it switches from primary to secondary. If you could post the full results of a scan again that'd be great.

Right now this means that using the values that you extracted from the beaconstore, we're able to generate all keys that the accessory is using and use them to fetch and decrypt their location reports. There are still a few issues though:

1. Secondary key generation appears to be broken, so reports can only be fetched until 4 am of when the accessory was "lost" (i.e. disconnected)
2. I don't know how to find the number of iterations to do in order to get the correct key for a certain time period. The key you posted was found at period 2, which is a bit weird given that it starts at 1 and the periods should take 15 minutes. I also don't know what happens when an accessory is "dead" for a few days; I assume the period continues where it left off, which means we cannot just do a rough (time in minutes / 15) to get the correct time. This probably just requires some experimentation.
3. I'm wondering whether it's possible to fetch the required accessory data straight from apple. I'll ask around a bit for this.

The library and example script have been updated; make sure to use the public key (so not the "lookup key") to make it work. It can now also read all the necessary details from the .plist file, so you don't need to fill out the keys manually anymore.

[malmeloo]

Nevermind, secondary key generation was actually working perfectly; I just made a silly mistake where it compared against the primary key twice. 🫠 Should be fixed now.

[YeapGuy]

Awesome :) It works for me with the first broadcasted key. But with the current public key, I get no match found.

Results of a scan now

Device - FC:1D:C1:E1:2A:25
  Public key:   /B3B4SolC42UdG4o73EbycccsWSWY+9+Ty7zJQ==
  Lookup key:   P1FJz8xpLGo9gusrbZSt93kyaGMAywRzyfymseGd+iE=
  Status byte:  20
  Hint byte:    f6
  Extra data:
    Adapter             : /org/bluez/hci0
    Address             : FC:1D:C1:E1:2A:25
    AddressType         : random
    Alias               : FC-1D-C1-E1-2A-25
    Blocked             : False
    Connected           : False
    LegacyPairing       : False
    ManufacturerData    : {76: bytearray(b'\x12\x19 \x0b\x8d\x94tn(\xefq\x1b\xc9\xc7\x1c\xb1d\x96c\xef~O.\xf3%\x03\xf6')}
    Paired              : False
    RSSI                : -60
    ServicesResolved    : False
    Trusted             : False
    UUIDs               : []

[malmeloo]

It works fine for me, but the key was found at index 192 / 200. That's 8 * 15 min = 2 hours ago, which also just happens to be exactly 2 days + 2 hours after you first paired the tag. The reason for this is that the library currently just generates the secondary key at number of time slots / 96 + 1, but this is wrong; the first update is at 4:00 AM, which is most likely less than 96 time slots (= 1 day) after it was paired, therefore the calculations don't line up.

I have just pushed an update that specifically generates keys for a certain timestamp. I'm uncertain whether the 4 AM rule is in UTC or the local timezone, but it should now be much more reliable.

[YeapGuy]

Yeah, it works with yesterday's key now. I'm not sure why it wasn't found before, yet it worked for you later... I reread your reply and I get it now. 😄

Today's key worked immediately.

The scan from today

Device - E1:0F:4D:EC:F9:70
  Public key:   oQ9N7Plwe+SIKnt5+k8GApo53nJzUcm8wAL+pA==
  Lookup key:   LG1q5ytNC9TmxBWHHX3mEfYm6LMLEjSxLZSQcxTJpb4=
  Status byte:  20
  Hint byte:    e
  Extra data:
    Adapter             : /org/bluez/hci0
    Address             : E1:0F:4D:EC:F9:70
    AddressType         : random
    Alias               : E1-0F-4D-EC-F9-70
    Blocked             : False
    Connected           : False
    LegacyPairing       : False
    ManufacturerData    : {76: bytearray(b'\x12\x19 {\xe4\x88*{y\xfaO\x06\x02\x9a9\xdersQ\xc9\xbc\xc0\x02\xfe\xa4\x02\x0e')}
    Paired              : False
    RSSI                : -46
    ServicesResolved    : False
    Trusted             : False
    UUIDs               : []

And the result of real_airtag.py

KEY FOUND!!
KEEP THE BELOW KEY SECRET! IT CAN BE USED TO RETRIEVE THE DEVICE'S LOCATION!
  - Key:           <removed>
  - Approx. Time:  2024-02-08 03:00:00+00:00
  - Type:          KeyType.SECONDARY

[hajekj]

I would like to share some of my work based on this library, which we used in the past week to attempt to recover a stolen MacBook. Unfortunately, it ran out of battery, before we put it all together and understood how it works. So we were unsuccessful, but I think there's quite a big potential with it, and could help others a lot: https://github.com/hajekj/OfflineFindRecovery

I plan to work on automating most of the steps, and making it into an application, so it can be used from phone or any device more easily.

[biemster]

The FindMy networks for AirTags (like this project) and for MacBooks / iPhones are two separate things if I quote the original seemoo research correctly. @hajekj are you querying Apple's servers with this to find your MacBook, or are you searching for the BLE beacon?

[hajekj]

I am doing both. Querying Apple service for historical movement data, and searching for BLE to find the precise location - like specific hotel room. It is the same thing in my opinion, just handles the key generation differently:

• MacBook / iPhone - only one key which changes every 15 minutes
• AirTags - uses the combination of both - when carried with owner, the public key changes every 15 minutes, when owner is not present, it changes every 24 hours to enable tracking detection when someone sticks it on you

[biemster]

I did not know that! I was under the impression that only airtags could be queried with the FindMy projects. It would be very handy to have a small script that dumps the private key from the login.keychain-db, just for safekeeping.
Putting this is on disk and not in the Secure Enclave seems like quite an oversight on Apple's side to me.

[hajekj]

Look into the repo there is a script to decrypt the keys of all the beacons you have - AirPods, MacBook, iPhone, AirTag etc.

[airy10]

https://gist.github.com/airy10/5205dc851fbd0715fcd7a5cdde25e7c8
will decrypt automlcally all the beacon files at once

Then copy the file from the OwnedBeacons directory for the device you want to "decrypted.plist" and you then can use the scripts from @hajekj to generate the keys using "findmy-keygeneration.py", then find the last known locations if you have an anisette server using "findmy-historicallocations.py"

Note the the script "findmy-keygeneration.py" needs some change for the AirTag (at least with my iTag chinese devices...) as the secondary key is from a different field :

secondary = device_data.get("secondarySharedSecret") or device_data.get("secureLocationsSharedSecret")
SKS = secondary["key"]["data"]

And the test
if key.key_type == KeyType.PRIMARY
should be removed
(and you might want to change the script too to generate only keys for the last month if you have some very old device)

With that, I could retrieve the history locations from my iTags

[hajekj]

I tried using that script, but couldn't get it to work @airy10 - I will try again and comment on the Gist, if I manage to isolate the issue, but I guess it could be MacOS version related.

[airy10]

It might be. It's fine here both from the command line or from a playground
Note that the first time you run it, you get some auth window about accessing the keychain to get the beacon password - so you might get an exception if you run it from some headless/ssh session

[samtombson]

I'm not clearly understand which PUBLIC_KEY should be used in real_airtag.py, if the only one I have is already contained in decrypted.plist, but the result there is no match found
I can't use device_scanner.py because the tag is far away

[wes1993]

@malmeloo,
Thanks a lot for your reply, i'll do some more tests in the next days and I'll write here what discover :-D

Another thing is that the script generates all the keys from the pairing days until today and next 48 hours so I should have all the keys independently from TZ but won't retrieve the locations..

Bye
Stefano

[wes1993]

@malmeloo,
I have seen in the WE that for my case i need to add 2hours to the timestamp reported.

If you need more details ask me :-D

[malmeloo]

Ah sorry, I missed the edit to your previous message. Can you share which script you are using?

As for that second issue, which reported timestamp are you referring to? The ones directly reported by this library (so the found-at and uploaded-at timestamps) are timezone-aware but default to UTC, which would explain the 2-hour difference. I'll fix this to default to the local timezone, but for now it can be fixed by calling .astimezone() on the datetime objects.

[wes1993]

Here are my scripts:

HistoricalLocations

import asyncio
import json
import logging
from pathlib import Path
import os
import csv
from datetime import datetime, timedelta, timezone
import json

from findmy import KeyPair
from findmy.reports import (
    AsyncAppleAccount,
    LoginState,
    RemoteAnisetteProvider,
    SmsSecondFactorMethod,
    TrustedDeviceSecondFactorMethod
)

# URL to (public or local) anisette server
ANISETTE_SERVER = "http://192.168.0.123:6969"


# Apple account details
ACCOUNT_EMAIL = ""
ACCOUNT_PASS = ""

logging.basicConfig(level=logging.DEBUG)


async def login(account: AsyncAppleAccount) -> None:
    state = await account.login(ACCOUNT_EMAIL, ACCOUNT_PASS)

    if state == LoginState.REQUIRE_2FA:  # Account requires 2FA
        # This only supports SMS methods for now
        methods = await account.get_2fa_methods()

        # Print the (masked) phone numbers
        for i, method in enumerate(methods):
            if isinstance(method, TrustedDeviceSecondFactorMethod):
                print(f"{i} - Trusted Device")
            elif isinstance(method, SmsSecondFactorMethod):
                print(f"{i} - SMS ({method.phone_number})")

        ind = int(input("Method? > "))

        method = methods[ind]
        await method.request()
        code = input("Code? > ")

        # This automatically finishes the post-2FA login flow
        await method.submit(code)

# Define a custom function to serialize datetime objects 
def serialize_datetime(obj): 
    if isinstance(obj, datetime): 
        return obj.isoformat() 
    raise TypeError("Type not serializable") 

async def fetch_reports(keys: list[KeyPair]) -> None:
    anisette = RemoteAnisetteProvider(ANISETTE_SERVER)
    acc = AsyncAppleAccount(anisette)

    try:
        acc_store = Path("account.json")
        try:
            with acc_store.open() as f:
                acc.restore(json.load(f))
        except FileNotFoundError:
            await login(acc)
            with acc_store.open("w+") as f:
                json.dump(acc.export(), f)

        print(f"Logged in as: {acc.account_name} ({acc.first_name} {acc.last_name})")

        # It's that simple!
        #print(keys)
        #print("\n")
        #print("AAAAAAAAAAAAAAA")
        reports = await acc.fetch_last_reports(keys)
        #print(reports)
        dump_list = []
        for keypair in reports:
            report = reports[keypair]
            for r in report:
                print("Pub_AT:", (r.published_at+timedelta(hours=2)), "----- TimeStamp:", (r.timestamp+ timedelta(hours=2)), "-----", r.latitude, r.longitude,"-----", r.key.private_key_b64)
                obj = {
                    "pub_at": (r.published_at+timedelta(hours=2)),
                    "time": (r.timestamp+ timedelta(hours=2)),
                    "lat": r.latitude,
                    "lon": r.longitude,
                    "published_at": r.published_at,
                    "description": r.description,
                    "confidence": r.confidence,
                    "status": r.status,
                    "key": r.key.private_key_b64
                }
                dump_list.append(obj)
                
                dbstring = str((r.published_at+timedelta(hours=2))) + ";" + str((r.timestamp+ timedelta(hours=2))) + ";" + str(r.latitude) + ";" + str(r.longitude) + ";" + str(r.key.private_key_b64) + "\n"
                with open('historylocations.txt', 'a', encoding="utf-8") as f:
                    f.write(str(dbstring))

        json_object = json.dumps(dump_list, indent=4, default=serialize_datetime)

        with open("location_history.json", "w") as outfile:
            outfile.write(json_object)

    finally:
        await acc.close()


if __name__ == "__main__":
    file = open('discovery-keys.csv', "r")
    csvreader = csv.reader(file, delimiter=";", quotechar='"', quoting=csv.QUOTE_ALL, lineterminator='\n')
    private_keys = []
    for row in csvreader:
        #print(row[0])
        private_keys.append(KeyPair.from_b64(row[2]))
        
    file.close()
    #print(private_keys)
    #FIX FOR WINDOWS COMMENT ON OTHER PLATFORM
    asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
    #END FIX FOR WINDOWS COMMENT ON OTHER PLATFORM
    asyncio.run(fetch_reports(private_keys))

FindKeys

"""
Example showing how to retrieve the primary key of your own AirTag, or any other FindMy-accessory.

This key can be used to retrieve the device's location for a single day.
"""
import plistlib
from datetime import datetime, timedelta, timezone
from pathlib import Path
#from csv import CSVWriter
import csv
from findmy.keys import KeyType

from findmy import FindMyAccessory

# Path to a .plist dumped from the Find My app.
PLIST_PATH = Path("airtag.plist")

# == The variables below are auto-filled from the plist!! ==

with PLIST_PATH.open("rb") as f:
    device_data = plistlib.load(f)

# PRIVATE master key. 28 (?) bytes.
MASTER_KEY = device_data["privateKey"]["key"]["data"][-28:]

# "Primary" shared secret. 32 bytes.
SKN = device_data["sharedSecret"]["key"]["data"]

# "Secondary" shared secret. 32 bytes.
# This doesn't apply in case of MacBook, but is used for AirTags and other accessories.
secondary = device_data.get("secondarySharedSecret") or device_data.get("secureLocationsSharedSecret")
SKS = secondary["key"]["data"]
#SKS = device_data["secureLocationsSharedSecret"]["key"]["data"]


def main() -> None:
    paired_at = device_data["pairingDate"].replace(tzinfo=timezone.utc)
    
    airtag = FindMyAccessory(MASTER_KEY, SKN, SKS, paired_at)

    # Generate keys for 2 days ahead
    now = datetime.now(tz=timezone.utc) + timedelta(hours=96)
    
    
    print()
    lookup_time = paired_at.replace(
        minute=paired_at.minute // 15 * 15,
        second=0,
        microsecond=0,
    ) + timedelta(minutes=15)

    #mycsv = CSVWriter('discovery-keys.csv')
    #mycsv = csv.writer("discovery-keys.csv")

#    while lookup_time < now:
#        keys = airtag.keys_at(lookup_time)
#        for key in keys:
#            if key.key_type == KeyType.PRIMARY:
#                print(lookup_time, key.adv_key_b64, key.private_key_b64, key.key_type, key.hashed_adv_key_b64)
#                #mycsv.write(lookup_time, key.adv_key_b64, key.private_key_b64, key.key_type, key.hashed_adv_key_b64)
#    if (timedelta((datetime.now(tz=timezone.utc) - timedelta(days=7) - lookup_time )) < 7):
#        print("Meno 7")
#    else:
#        print("Piu 7")
    
#    print(datetime.now(tz=timezone.utc) - timedelta(days=7))
#    print (lookup_time(tz=timezone.utc))
    #lookup_time = datetime.now(tz=timezone.utc) - timedelta(days=7)
    while lookup_time < now:
        keys = airtag.keys_at(lookup_time)
        for key in keys:
            #print(key.key_type)
            #if (str(key.key_type) == "KeyType.SECONDARY"):
            #if (str(key.key_type) == "KeyType.PRIMARY"):
            if True:
                with open("discovery-keys.csv", 'a') as csvfile:
                    csvwriter = csv.writer(csvfile, delimiter=";", lineterminator='\n')
                    csvwriter.writerow([lookup_time, key.adv_key_b64, key.private_key_b64, key.key_type, key.hashed_adv_key_b64])
            #print(lookup_time, key.adv_key_b64, key.private_key_b64, key.key_type, key.hashed_adv_key_b64)
            #mycsv.write(lookup_time, key.adv_key_b64, key.private_key_b64, key.key_type, key.hashed_adv_key_b64)
                
        lookup_time += timedelta(minutes=15)

    print("All keys for specified time period generated")
if __name__ == "__main__":
    open('discovery-keys.csv', 'w').close()
    main()

Best Regards
Stefano

[malmeloo]

With #26 merged, I've updated real_airtag.py to generate keys for the past week and fixed the timezone issue. It also addresses another issue which I believe may have been the cause of the limited number of reports.

Could you try that example now and see if it works for you? Make sure to actually run the module from main, as these changes haven't officially been released yet.

[wes1993]

Hello @malmeloo,
I have tested your script and seems that works well, do you have some suggestion how can we send this to homeassistant?
To track this on a map :-D

Thanks for your work!!

[malmeloo]

That's nice to hear! I've actually already been working on exactly that, so stay tuned ;-).

[wes1993]

Hahaha perfect :-D

Just one thing, I have seen that some locations is missing here an example:

But as per FindMy app I'm sure that we should have a location at 10:56

Some ideas?

I have seen that we won't retrieve all the locations, probably we need more keys?

Again thanks a lot for your works!!! :-D

[wes1993]

Another example (Tested now):

Locations from script

[malmeloo]

Are you sure the airtag wasn't in range at that time? I think the FindMy app also shows those locations, though I'm not 100% sure.

I'll take another look at the implementation, maybe the key generator is indeed missing some keys.

[wes1993]

@malmeloo,
Thanks for support!! :)

If you refer to location for this night, the AirTag is in range with iPhone, that I have used to see findmy app

Do you think that if the AirTag is in range the findmy won't update the location to apple servers?

[malmeloo]

Indeed, to my knowledge the AirTag does not broadcast any keys when the owner device has recently connected to it, because it is in nearby or connected state. I think that in this case, the FindMy app simply shows the location of the phone when it last connected to your AirTag. In that case the location history shown in the app is not exclusively sourced from the actual FindMy network, which is why this library could be missing some location reports.

[faceless2]

@malmeloo thanks for all your work on this. I've just started with FindMy/airtags and have been testing your latest real_airtag.py, am working with the current code from git, using a new airtag I paired yesterday.

The problem I'm hitting is that this exception:

  File "/home/mike/FindMy.py/examples/findmy/reports/reports.py", line 221, in fetch_reports
    reports.extend(await self._fetch_reports(date_from, date_to, chunk))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/FindMy.py/examples/findmy/reports/reports.py", line 252, in _fetch_reports
    reports.append(LocationReport.from_payload(key, date_published, description, payload))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/FindMy.py/examples/findmy/reports/reports.py", line 133, in from_payload
    data = _decrypt_payload(payload, key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/FindMy.py/examples/findmy/reports/reports.py", line 26, in _decrypt_payload
    eph_key = ec.EllipticCurvePublicKey.from_encoded_point(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/.local/lib/python3.12/site-packages/cryptography/hazmat/primitives/asymmetric/ec.py", line 180, in from_encoded_point
    raise ValueError("Unsupported elliptic curve point type")

Checking the payload bytes, I have about 30 responses that look like this:

2b d8 fa f9 07 04 c1 ce 6f ...

which are decoding fine. Then I get a response like this, which is failing

2b d8 e6 45 00 05 04 de ...

byte 5 changes from 04 to 05. I know nothing about this response but presuming it's ASN.1, byte 04 indicates a string, byte 05 indicates null.

Any ideas? Is the full reponse string safe to share here? Or if you can point me to some details on the format of this string I'll try and parse it and get you some more information.

[faceless2]

Following myself up, it looks like if I just check for payload[5] == 5 and skip that response if true, nothing breaks and I can decode the remaining responses - so I'm now getting lats/lons reported. Very pleased!

EDIT: I had to make another small change in real_airtag.py to handle recently paired devices:

fetch_from = max(PAIRED_AT, fetch_to - timedelta(days=7))

[malmeloo]

@faceless2 very interesting... I do indeed believe that it's ASN.1, but I'm also not entirely certain. I wonder why it would be null though? That part of the payload encodes the ephemeral key used to encrypt the location report, so it makes no sense to me... unless the report is unencrypted? Is the payload's length any different compared to the other ones?

It looks like you're on the right track though, judging by cryptography's source code. Assuming it is indeed ASN.1 encoded, the library is only able to decode integers and bit/octet strings, which makes sense. Would you be willing to share that payload? That should be safe to do, it's considered public information according to the network protocol. The key still needs to be DH'd with your private key in order to decrypt its contents.

And thank you for the fix, I'll update the example in a second 🙂

[airy10]

Hello @malmeloo, I have tested your script and seems that works well, do you have some suggestion how can we send this to homeassistant? To track this on a map :-D

Thanks for your work!!

You can use MQTT or Home Assistant devicetracker "see" service for that.
That's what I'm doing with my FindMyDevices Swift app (while it's only using local Find My app files info for now, not asking Apple servers yet)
That's Swift code but you might find some interesting starting point info here :
https://github.com/airy10/FindMyDevices/blob/main/FindMyDevices/DevicesManager.swift

(updateMQTT function for MQTT, and updateHomeAssistant for "/api/services/device_tracker/see" service)
You can check Home Assistant doc for more details

Airy

[faceless2]

@malmeloo here's the payload:

2bd8e645000504dee6a88ab4580e4c90dcacc62e0efae57a003a1bac1212ba670398be61cb648e08417cf3714ee0429f961793aa645600a7f69807e215292da3a6bda511953cb15b4840de16e26c17651eef760e77fa4b2ca5

Things have moved on in the last hour (!) and I've now got this working with three airtags (I modified your real_airtag.py to take a list of "plist" files and retrieve location data for each one). One tag was added yesterday evening, two in the last few hours but I am only seeing this single message with a byte 5 in that location. The payload length looks about the same, which is also odd - if one contains an EC key and one contains null, the second should be much shorter. In addition although the byte after that is a 4, indicating an ASN.1 octet string, I can't decode it as one.

Finally, I realise this is a quick proof of concept you only added last night but I already have some very minor suggestions, if you don't mind.

• it's useful being able to pass in a list of plist files in argv if you have more than one device.
• I see you can set a name on the device but it would be nice if that came through in the report too - useful with multiple airtags.

[wes1993]

Hello @malmeloo, I have tested your script and seems that works well, do you have some suggestion how can we send this to homeassistant? To track this on a map :-D
Thanks for your work!!

You can use MQTT or Home Assistant devicetracker "see" service for that. That's what I'm doing with my FindMyDevices Swift app (while it's only using local Find My app files info for now, not asking Apple servers yet) That's Swift code but you might find some interesting starting point info here : https://github.com/airy10/FindMyDevices/blob/main/FindMyDevices/DevicesManager.swift

(updateMQTT function for MQTT, and updateHomeAssistant for "/api/services/device_tracker/see" service) You can check Home Assistant doc for more details

Airy

Hello @airy10,
I have seen the devicetracker "see", but seems that we can't send a specific datetime in addition to the Lat and Lon is this correct?
The correct approach should be to send Lat/Lon and at what Time (Taken from findmy library)

Do you know a way to send also date and time?

Best regards
Stefano

[malmeloo]

@faceless2 Thanks! I've created a new issue to track this, as I'm not particularly convinced this is an AirTag-only problem: #27
I'm erring on the side of "Apple messed up" here, but in any case, raising an exception is not the right response in this situation.

As for your other suggestions, I like the idea of passing a plist in as an argument, although I would like to keep the examples as simple as possible. In my experience they quickly get too convoluted, but I'd like them to only really show the core functionality of a certain task so that people can learn from them quickly.

The friendly name is not really possible unfortunately, since this information is not collected by Apple in the location report. I have just finished a change that allows you to quickly query reports for an accessory through the library itself, avoiding much of the key logic that is currently present in the example. That should also make it easier to distinguish reports in any code using it.

[airy10]

Hello @airy10, I have seen the devicetracker "see", but seems that we can't send a specific datetime in addition to the Lat and Lon is this correct? The correct approach should be to send Lat/Lon and at what Time (Taken from findmy library)

Do you know a way to send also date and time?

Best regards Stefano

Unfortunately, I don't think that there is a way to force the date associated with the lat/long, either with MQTT or the see device. You can add send some custom attributes but they won't be used by "map". I think that some device_tracker subclass let the user set some custom update date so adding some new subclass might be a way.
Or maybe use some template to extract the additional attribute for that. I guess I might try that :)

[wes1993]

@airy10,
Yes because we run the software but everytime we run the software we don't receive an updated location so we can't simple send the Lat/Lon to homeasssistan

[faceless2]

Last suggestion for this issue: I can confirm that with a small change, the real_airtag.py script works with plists relating to iphones, as well as airtags (I followed the guide from @hajekj at https://github.com/hajekj/OfflineFindRecovery to identify the correct device).

The only change required was

if "secondarySharedSecret" in device_data :
    SKS = device_data["secondarySharedSecret"]["key"]["data"]
else:
    SKS = device_data["secureLocationsSharedSecret"]["key"]["data"]

well, that and a try/catch around _decrypt_payload as suggested in #27

[malmeloo]

@faceless2 Great feedback, thank you! I've integrated plist parsing directly into the library now, and it should now also support iPhones with that change. The examples now also read the keys or plist path directly from the command line arguments.

For now these changes are only present in #28, but I intend to merge that soon and make a new, proper release shortly after.

[malmeloo]

As of a few minutes ago, official accessories are now officially supported! 🎉v0.6.0 has been published to PyPi, so it is no longer necessary to run the development version for this functionality. The examples have also been updated to include these changes in the library.

Since this is now implemented, I am going to close this issue. If you're running into problems while using this feature, feel free to open a new one!

[wes1993]

@malmeloo,
I'm not sure is this is the correct place but if you want you can add this lines in the real_airtag example so the list is printed cronologically instead of in casual order:

"""
Example showing how to fetch locations of an AirTag, or any other FindMy accessory.
"""
from __future__ import annotations

import logging
import sys
from pathlib import Path

from _login import get_account_sync

from findmy import FindMyAccessory
from findmy.reports import RemoteAnisetteProvider

# URL to (public or local) anisette server
ANISETTE_SERVER = "http://localhost:6969"

logging.basicConfig(level=logging.INFO)


def main(plist_path: str) -> int:
    # Step 0: create an accessory key generator
    with Path(plist_path).open("rb") as f:
        airtag = FindMyAccessory.from_plist(f)

    # Step 1: log into an Apple account
    print("Logging into account")
    anisette = RemoteAnisetteProvider(ANISETTE_SERVER)
    acc = get_account_sync(anisette)

    # step 2: fetch reports!
    print("Fetching reports")
    reports = acc.fetch_last_reports(airtag)

    # step 3: print 'em
    print()
    print("Location reports:")
    sorted_reports = sorted(reports, key=lambda x: x.timestamp)
    
    for sorted_report in sorted_reports:
        print(f" - {sorted_report}")

    return 0


if __name__ == "__main__":
    if len(sys.argv) < 2:
        print(f"Usage: {sys.argv[0]} <path to accessory plist>", file=sys.stderr)
        print(file=sys.stderr)
        print("The plist file should be dumped from MacOS's FindMy app.", file=sys.stderr)
        sys.exit(1)

    sys.exit(main(sys.argv[1]))

[malmeloo]

@wes1993 Thanks, updated!