Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ford: use platform codes to fuzzy fingerprint #31124

Merged
merged 58 commits into from
May 8, 2024

Conversation

incognitojam
Copy link
Contributor

@incognitojam incognitojam commented Jan 23, 2024

Use the FW prefix to identify the model and model year/generation of the vehicle.

Notes:

  • This ignores the "software version" suffix of the FW since we can identify changes in API (e.g. Ford Q3 -> Q4) from the rest of the FW. For example, the 2024 Escape is Q4 and its FW has a new model year hint (see FW table)
  • The FW version cannot tell us whether the car is compatible (has LCA/ACC), except the lack of a radar can be used to rule cars out. Some cars have radar without ACC, and some with ACC do not have LCA.
  • We can identify the model from the 'platform hint' and model year/generation from the 'model year hint' without any false matches/collisions (see notebook using big vin database)

Verification

Future improvements

  • Remove requirement for all four ECUs to match: we can guarantee a match/no false positives from just three for some platforms

May close #31052

Platform codes (platform + model year hints)
FORD BRONCO SPORT 1ST GEN
  (Ecu.eps, 0x730, None):
    Codes: [(b'X6C', b'L')]
  (Ecu.abs, 0x760, None):
    Codes: [(b'X6C', b'L')]
  (Ecu.fwdRadar, 0x764, None):
    Codes: [(b'B5T', b'L')]
  (Ecu.fwdCamera, 0x706, None):
    Codes: [(b'1PT', b'M')]

FORD ESCAPE 4TH GEN
  (Ecu.eps, 0x730, None):
    Codes: [(b'X6C', b'L')]
  (Ecu.abs, 0x760, None):
    Codes: [(b'X6C', b'L')]
  (Ecu.fwdRadar, 0x764, None):
    Codes: [(b'B5T', b'L')]
  (Ecu.fwdCamera, 0x706, None):
    Codes: [(b'J6T', b'L'), (b'V4T', b'L')]

FORD EXPLORER 6TH GEN
  (Ecu.eps, 0x730, None):
    Codes: [(b'1MC', b'L'), (b'1MC', b'M'), (b'1MC', b'P')]
  (Ecu.abs, 0x760, None):
    Codes: [(b'1MC', b'L')]
  (Ecu.fwdRadar, 0x764, None):
    Codes: [(b'B5T', b'L')]
  (Ecu.fwdCamera, 0x706, None):
    Codes: [(b'B5T', b'L'), (b'C5T', b'L')]

FORD F-150 14TH GEN
  (Ecu.eps, 0x730, None):
    Codes: [(b'L3V', b'M')]
  (Ecu.abs, 0x760, None):
    Codes: [(b'L34', b'P')]
  (Ecu.fwdRadar, 0x764, None):
    Codes: [(b'L3T', b'M')]
  (Ecu.fwdCamera, 0x706, None):
    Codes: [(b'J6T', b'P')]

FORD F-150 LIGHTNING 1ST GEN
  (Ecu.abs, 0x760, None):
    Codes: [(b'L38', b'P')]
  (Ecu.fwdRadar, 0x764, None):
    Codes: [(b'L3T', b'M')]
  (Ecu.fwdCamera, 0x706, None):
    Codes: [(b'L3T', b'M')]

FORD MUSTANG MACH-E 1ST GEN
  (Ecu.eps, 0x730, None):
    Codes: [(b'J9C', b'L')]
  (Ecu.abs, 0x760, None):
    Codes: [(b'K9C', b'L')]
  (Ecu.fwdRadar, 0x764, None):
    Codes: [(b'L3T', b'M')]
  (Ecu.fwdCamera, 0x706, None):
    Codes: [(b'L3T', b'M')]

FORD FOCUS 4TH GEN
  (Ecu.eps, 0x730, None):
    Codes: [(b'X6C', b'J')]
  (Ecu.abs, 0x760, None):
    Codes: [(b'X61', b'J')]
  (Ecu.fwdRadar, 0x764, None):
    Codes: [(b'X7T', b'J')]
  (Ecu.fwdCamera, 0x706, None):
    Codes: [(b'X7T', b'J')]

FORD MAVERICK 1ST GEN
  (Ecu.eps, 0x730, None):
    Codes: [(b'Z6C', b'N')]
  (Ecu.abs, 0x760, None):
    Codes: [(b'Z6C', b'N'), (b'Z6C', b'P')]
  (Ecu.fwdRadar, 0x764, None):
    Codes: [(b'Z6T', b'N')]
  (Ecu.fwdCamera, 0x706, None):
    Codes: [(b'Z6T', b'N')]


Car models vs. platform codes:
  (Ecu.eps, 0x730, None):
    (b'X6C', b'L'): ['FORD BRONCO SPORT 1ST GEN', 'FORD ESCAPE 4TH GEN']
    (b'1MC', b'P'): ['FORD EXPLORER 6TH GEN']
    (b'1MC', b'M'): ['FORD EXPLORER 6TH GEN']
    (b'1MC', b'L'): ['FORD EXPLORER 6TH GEN']
    (b'L3V', b'M'): ['FORD F-150 14TH GEN']
    (b'J9C', b'L'): ['FORD MUSTANG MACH-E 1ST GEN']
    (b'X6C', b'J'): ['FORD FOCUS 4TH GEN']
    (b'Z6C', b'N'): ['FORD MAVERICK 1ST GEN']
  (Ecu.abs, 0x760, None):
    (b'X6C', b'L'): ['FORD BRONCO SPORT 1ST GEN', 'FORD ESCAPE 4TH GEN']
    (b'1MC', b'L'): ['FORD EXPLORER 6TH GEN']
    (b'L34', b'P'): ['FORD F-150 14TH GEN']
    (b'L38', b'P'): ['FORD F-150 LIGHTNING 1ST GEN']
    (b'K9C', b'L'): ['FORD MUSTANG MACH-E 1ST GEN']
    (b'X61', b'J'): ['FORD FOCUS 4TH GEN']
    (b'Z6C', b'P'): ['FORD MAVERICK 1ST GEN']
    (b'Z6C', b'N'): ['FORD MAVERICK 1ST GEN']
  (Ecu.fwdRadar, 0x764, None):
    (b'B5T', b'L'): ['FORD BRONCO SPORT 1ST GEN', 'FORD ESCAPE 4TH GEN', 'FORD EXPLORER 6TH GEN']
    (b'L3T', b'M'): ['FORD F-150 14TH GEN', 'FORD F-150 LIGHTNING 1ST GEN', 'FORD MUSTANG MACH-E 1ST GEN']
    (b'X7T', b'J'): ['FORD FOCUS 4TH GEN']
    (b'Z6T', b'N'): ['FORD MAVERICK 1ST GEN']
  (Ecu.fwdCamera, 0x706, None):
    (b'1PT', b'M'): ['FORD BRONCO SPORT 1ST GEN']
    (b'J6T', b'L'): ['FORD ESCAPE 4TH GEN']
    (b'V4T', b'L'): ['FORD ESCAPE 4TH GEN']
    (b'B5T', b'L'): ['FORD EXPLORER 6TH GEN']
    (b'C5T', b'L'): ['FORD EXPLORER 6TH GEN']
    (b'J6T', b'P'): ['FORD F-150 14TH GEN']
    (b'L3T', b'M'): ['FORD F-150 LIGHTNING 1ST GEN', 'FORD MUSTANG MACH-E 1ST GEN']
    (b'X7T', b'J'): ['FORD FOCUS 4TH GEN']
    (b'Z6T', b'N'): ['FORD MAVERICK 1ST GEN']

TODO: write scripts/tests (print platform codes and version ranges etc.)

May close commaai#31052
@github-actions github-actions bot added car vehicle-specific ford labels Jan 23, 2024
@sshane
Copy link
Contributor

sshane commented Jan 26, 2024

great work! I love how this is mostly just our other fuzzy FP functions; it mostly works for all brands. With verification, I made sure that:

  • No dongles ever changed fingerprint except mock (including checking all other brands), running match_fw_to_car
  • Check how many mock dongles gained a fingerprint (match_fw_to_car) @jnewb1 can we add mock unfingerprinting segments to the dataset if not already there?
  • Check how many Ford fingerprinting dongles (fuzzy or exact) gained a fingerprint, running match_fw_to_car(allow_exact=False)
  • No dongles lost a fingerprint (checking both allow_exact True and False

Let me know if you have any other verification step ideas

@incognitojam
Copy link
Contributor Author

incognitojam commented Jan 27, 2024

I found some VINs for Ford Explorers online (most of our FW PRs are for those) and used https://motorcraft.com/AsBuilt to download information about them (ECU configuration, part numbers and FW versions). I collected 44 VINs and data.

Here are the results of running the current OP fingerprint vs running with the custom fuzzy matcher:

Show results

changes by vin
model year stats
total stats

Of the 15 newly fingerprinted VINs, 100% were previously failing to fingerprint due to missing Engine FWs. For example, LB5A-14C204-EAE was not in the offline database, but there are many versions with the same platform code B5A and part number 14C204, so the custom matcher allows it to match.

Of the 21 VINs which continue to not match, three of these (MY17 and MY19) did not have any camera or radar part numbers/fw. The other 18 did have cameras but did not have radars.

@sshane
Copy link
Contributor

sshane commented Jan 27, 2024

Of the 15 newly fingerprinted VINs, 100% were previously failing to fingerprint due to missing Engine FWs. For example, LB5A-14C204-EAE was not in the offline database, but there are many versions with the same platform code B5A and part number 14C204, so the custom matcher allows it to match.

I wonder how many matches we gain just by removing engine!

Of the 21 VINs which continue to not match, three of these (MY17 and MY19) did not have any camera or radar part numbers/fw. The other 18 did have cameras but did not have radars.

So with a perfect fuzzy matcher, we still expect none of these to match? Ie. they all have LKAS but not ACC as stock?

@incognitojam
Copy link
Contributor Author

incognitojam commented Jan 27, 2024

So with a perfect fuzzy matcher, we still expect none of these to match? Ie. they all have LKAS but not ACC as stock?

Yes pretty much.

Looking at the cars which are fingerprinting in more detail, there is a car which shouldn't be compatible which is matching right now, without the custom fuzzy matcher.

For example this 2020 Ford Explorer XLT (1FMSK8DH1LGB19569) has this FW:

Ecu.eps: [b'L1MC-14D003-AK\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00']
Ecu.abs: [b'L1MC-2D053-BA\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00']
Ecu.fwdCamera: [b'LB5T-14F397-AE\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00']
Ecu.engine: [b'LB5A-14C204-BUJ\x00\x00\x00\x00\x00\x00\x00\x00\x00']

Note it doesn't have a radar. It fingerprints anyway, I think because fuzzy matching requires just two matching ECUs.

Fingerprinted FORD EXPLORER 6TH GEN using fuzzy match. 2 matching ECUs

Since camera/radar/eps are excluded from fuzzy, it's the abs and engine FW matching the database. Both were added in fd2e03d.

Looking at that PR, the 2020 Explorer Limited (1FMSK8FH6LGB17698) does have a radar, and all other FW are identical. So either we have to require the radar FW or use some other query to determine whether it has ACC.

Ecu.eps: [b'L1MC-14D003-AK\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00']
Ecu.abs: [b'L1MC-2D053-BA\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00']
Ecu.fwdRadar: [b'LB5T-14D049-AB\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00']
Ecu.fwdCamera: [b'LB5T-14F397-AE\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00']
Ecu.engine: [b'LB5A-14C204-BUJ\x00\x00\x00\x00\x00\x00\x00\x00\x00']
Show full results
                   VIN                         Comment            Fingerprint  Fuzzy         NewFingerprint  NewFuzzy      FactoryACC FactoryTJA        Result
23  1FMSK8DHXLGBXXXXXX          2020 Ford Explorer XLT  FORD EXPLORER 6TH GEN  False  FORD EXPLORER 6TH GEN     False     RadarFusion         On      fp -> fp
25  1FM5K8GC0LGAXXXXXX           2020 Ford Explorer ST  FORD EXPLORER 6TH GEN  False  FORD EXPLORER 6TH GEN     False     RadarFusion         On      fp -> fp
39  1FM5K8GC3LGAXXXXXX           2020 Ford Explorer ST  FORD EXPLORER 6TH GEN  False  FORD EXPLORER 6TH GEN     False     RadarFusion         On      fp -> fp
41  1FMSK8DH0NGBXXXXXX      2022 Ford Explorer XLT 4WD  FORD EXPLORER 6TH GEN  False  FORD EXPLORER 6TH GEN     False     RadarFusion         On      fp -> fp
6   1FMSK8DH1LGBXXXXXX          2020 Ford Explorer XLT  FORD EXPLORER 6TH GEN   True  FORD EXPLORER 6TH GEN      True  Off (no radar)        Off      fp -> fp
8   1FMSK8DH7NGBXXXXXX          2022 Ford Explorer XLT  FORD EXPLORER 6TH GEN   True  FORD EXPLORER 6TH GEN      True  Off (no radar)        Off      fp -> fp
10  1FMSK8DH6NGBXXXXXX          2022 Ford Explorer XLT  FORD EXPLORER 6TH GEN   True  FORD EXPLORER 6TH GEN      True  Off (no radar)        Off      fp -> fp
17  1FMSK8DH9NGBXXXXXX          2022 Ford Explorer XLT  FORD EXPLORER 6TH GEN   True  FORD EXPLORER 6TH GEN      True  Off (no radar)        Off      fp -> fp
11  1FMSK8FH7NGAXXXXXX      2022 Ford Explorer Limited                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
12  1FMSK7FHXPGAXXXXXX  2023 Ford Explorer Limited RWD                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
15  1FMSK8DH9LGBXXXXXX      2020 Ford Explorer XLT 4WD                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
20  1FMSK8FH9MGAXXXXXX  2021 Ford Explorer Limited 4WD                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
21  1FMSK8DH1LGBXXXXXX          2020 Ford Explorer XLT                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
22  1FMSK8DH1LGDXXXXXX          2020 Ford Explorer XLT                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
24  1FMSK8DHXLGAXXXXXX          2020 Ford Explorer XLT                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
26  1FMSK8FH2MGBXXXXXX      2021 Ford Explorer Limited                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
27  1FMSK8FHXMGBXXXXXX      2021 Ford Explorer Limited                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
28  1FMSK8FH2MGBXXXXXX      2021 Ford Explorer Limited                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
29  1FMSK8FH8MGBXXXXXX      2021 Ford Explorer Limited                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
31  1FMSK8FH5MGAXXXXXX      2021 Ford Explorer Limited                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
32  1FMSK8FH4MGBXXXXXX      2021 Ford Explorer Limited                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
33  1FMSK8FH5MGAXXXXXX      2021 Ford Explorer Limited                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
38  1FMSK8DH5MGAXXXXXX          2021 Ford Explorer XLT                   mock  False  FORD EXPLORER 6TH GEN      True     RadarFusion         On    mock -> fp
0   1FMSK8DH1NGAXXXXXX          2022 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
1   1FMSK8DH7MGCXXXXXX          2021 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
2   1FMSK8DH9MGBXXXXXX          2021 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
3   1FMSK8DH4NGAXXXXXX          2022 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
4   1FMSK8DH8MGAXXXXXX          2021 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
5   1FMSK8DH3NGAXXXXXX          2022 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
7   1FMSK8DH8NGAXXXXXX          2022 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
9   1FMSK8DH9LGBXXXXXX          2020 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
13  1FMSK8DH0NGAXXXXXX          2022 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
14  1FMSK8DH9LGCXXXXXX          2020 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
16  1FMSK8DH6NGBXXXXXX          2022 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
18  1FMSK8DH0NGAXXXXXX          2022 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
19  1FM5K8D83KGAXXXXXX          2019 Ford Explorer XLT                   mock  False                   mock     False       no camera  no camera  mock -> mock
30  1FMSK8DH1LGCXXXXXX          2020 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
34  1FMSK7DH5LGAXXXXXX          2020 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
35  1FM5K8D8XHGCXXXXXX      2017 Ford Explorer XLT 4WD                   mock  False                   mock     False       no camera  no camera  mock -> mock
36  1FMSK8DH0LGBXXXXXX          2020 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
37  1FMSK8DH4MGAXXXXXX              2021 Ford Explorer                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
40  1FM5K8GT6HGDXXXXXX        2017 Ford Explorer Sport                   mock  False                   mock     False       no camera  no camera  mock -> mock
42  1FMSK8DH1NGAXXXXXX          2022 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock
43  1FMSK8DH5NGAXXXXXX          2022 Ford Explorer XLT                   mock  False                   mock     False  Off (no radar)        Off  mock -> mock

@sshane
Copy link
Contributor

sshane commented Jan 28, 2024

I think we should pick a subset of ECUs to care about, and ignore the rest. This random Bronco does not fingerprint because the engine version is out of range (we pick radar and camera on Hyundai and radar camera and EPS on Toyota): d1aacbb70e8a9d4b|2024-01-27--09-47-46

@sheaduncan
Copy link
Contributor

A heads up, alot of us are running this UDS fix commaai/panda#1524 to get CANFD responses.

I'm not sure you've tested this with some Mazda data, but I found shared ECUs.
commaai/panda#1524 (comment)

@incognitojam incognitojam marked this pull request as ready for review March 9, 2024 17:34
@@ -0,0 +1,30 @@
#!/usr/bin/env python3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a common script that takes brand eventually

@sshane sshane merged commit 85c9fea into commaai:master May 8, 2024
27 checks passed
@incognitojam incognitojam deleted the ford-platform-codes branch May 9, 2024 18:49
cydia2020 pushed a commit to cydia2020/dodgypilot that referenced this pull request May 14, 2024
* Ford: use platform codes to fuzzy fingerprint

TODO: write scripts/tests (print platform codes and version ranges etc.)

May close commaai#31052

* get_platform_codes: fix return type

* add print_platform_codes.py script

* print_platform_codes: sort versions

* match_fw_to_car_fuzzy: use set comprehension, and fix typo

* Ford: add missing Mach-E fw

From the route 83a4e056c7072678/2023-11-13--16-51-33 (which is already
in selfdrive/car/tests/routes.py, added in commaai#30691).

* add ford_fuzzy_fingerprint.ipynb notebook

* get_platform_codes: use regex to parse firmware

* test_ford: test_platform_codes_fuzzy_fw

* test_ford: use get_platform_codes in test_fw_versions

* match_fw_to_car_fuzzy: improve comments

* test_ford: add test_platform_codes_spot_check

* test_ford: add test_match_fw_fuzzy

* remove comment from notebook

* TestFordFW: remove engine ECU FW

* update print_platform_codes.py

* remove part number (unecessary)

* platform codes can just use platform hint and model year - software revision not useful

* fuzzy FP on the platform hint and model year hint range

* fix platform codes test

* update notebook

* add notebook

* explain model year hint better

* test part numbers again

* cleanup notebooks

* remove notebook

* cleanup match_fw_to_car_fuzzy and add comments

* update comment

* .

* Revert "remove notebook"

This reverts commit 5d4ca20.

* add notebook back

* remove PSCM from PLATFORM_CODE_ECUS

ABS and IPMA are the best for uniquely matching, and the radar is always required

* Revert "remove PSCM from PLATFORM_CODE_ECUS"

This reverts commit b7baeac.

* fix from merge

* more fixes

revert

* FW_RE -> FW_PATTERN

* this can actually be set

* conventions

* just add

* convention

* refactor matcher, this brings it more in line with Hyundai. IMPORTANT NOTE:

NOTE THAT WE remove the separation for the different platform code model year hint ranges, I don't see that being a problem

* better/smaller test

* add test to catch overlapping platform codes

* remove nb

* not now

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
car vehicle-specific ford
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[$500 bounty] Ford: reliable fuzzy fingerprinting
3 participants