Update Samsung phone names

[mr-bobdobalina]

Mostly cosmetic and maybe my OCD kicking in. But would it be possible to update the Samsung camera names to their actual commercially known names? I'm aware that the names are being pulled directly from the EXIF data, but it would be nice if perhaps Photoprism could convert something like 'Samsung SM-F936U1' into 'Samsung Galaxy Z Fold4' (The actual phone name most people recognize)

[lastzero]

https://github.com/photoprism/photoprism/blob/develop/internal/entity/camera_makes.go

[graciousgrey]

In case you cannot create a pull request you can post a list with cameras names and their related commercial name here, so that we can add it.

[srett]

Should we maybe gather some data before creating a PR, to avoid making a dozen small ones? (I'd be willing to do that but also have a few questions about convention before starting.)

Here's a one-liner to dump all your cameras with make and model, in case anyone wants to contribute:

console.log(__CONFIG__.cameras.reduce((carry, item) => carry + "\n" + '"' + item.Make + '" "' + item.Model + '"', ''))

Open PP in browser, then run it in the JS console (Ctrl-Shift-I). I saw I have a couple of unidentified ones too.

Here's a list of easy candidates from me:

"HUAWEI" "C8817E" -> This was from the "Honor" Series, but I don't remember the name :-/ Might've been "Honor Play 4"
"SAMSUNG" "GT-I9300" -> Galaxy S3
"Samsung" "SM-G950F" -> Galaxy S8
"Samsung" "SM-G965F" -> Galaxy S9+

The following ones are a bit odd, e.g. Kodak and Nikon repeat the manufacturer name in the model string. Should this be fixed up too? If yes, simply by adding each model to the lookup table, or maybe by some more general approach? For example, having a set of manufacturer names that get stripped from the model name, a la model = model.replace("^(KODAK|NIKON)\s+", ""). Same goes for Kodak's weird "DIGITAL CAMERA" suffix. I think it's ugly and would remove it, what are your thoughts here?
Also the manufacturer names could be shortened to just Kodak, Nikon, Casio, Medion.

"EASTMAN KODAK COMPANY" "KODAK EASYSHARE CX7300 DIGITAL CAMERA"
"EASTMAN KODAK COMPANY" "KODAK V550 ZOOM DIGITAL CAMERA"
"EASTMAN KODAK COMPANY" "KODAK V570 DUAL LENS DIGITAL CAMERA"
"NIKON CORPORATION" "NIKON D3000"
"NIKON CORPORATION" "NIKON D80"
"CASIO COMPUTER CO.,LTD." "EX-S600"
"MEDION" "5M DIGICAM"
"Medion" "636Z3"
"MEDION OPTICAL CO,LTD" "MD411Z3"

Finally some oddball. Hard to believe it was sold with that model name slapped on the actual thing; googling it results in almost nothing... Anyone?

"Olympus" "uT6000,ST6000"

Also, what about normalizing the capitalization? SAMSUNG vs. Samsung....

[srett]

Aigtht, will do. Any thoughts about the repetition of the company name in the model name, i.e. these five:

"EASTMAN KODAK COMPANY" "KODAK EASYSHARE CX7300 DIGITAL CAMERA"
"EASTMAN KODAK COMPANY" "KODAK V550 ZOOM DIGITAL CAMERA"
"EASTMAN KODAK COMPANY" "KODAK V570 DUAL LENS DIGITAL CAMERA"
"NIKON CORPORATION" "NIKON D3000"
"NIKON CORPORATION" "NIKON D80"

So after normalizing e.g. "NIKON CORPORATION" to "Nikon", the full name as displayed in the UI would be "Nikon NIKON D80". Would you prefer adding some code like the regex suggestion above later on, or should I just special-case those five via the lookup table for now?

[lastzero]

Not 100% sure, but I remember to have added some code that automatically detects and fixes this. If not, it might be worth implementing.

[srett]

I just found something potentially really useful: Wikipedia apparently already has a (huge?) database of mapping exif make/model to the proper human readable variant, see for example Kodak EasyShare CX7300. The infobox to the right contains the EXIF data, and the friendly/human readable form of both fields. If you click 'edit' and check the page source, you'll find that the data seems to get pulled from wikidata, so it might be easy to extract it all and build a comprehensive lookup table. Afaik Wikidata is CC0 aka public domain, so no licensing issues either. Just not sure if it's a good idea at that point to create a huge .go source file with that, instead of loading a big JSON file at runtime.

[lastzero]

You can have both with go generate. See txt and migrate packages for examplea. This way you have native Go code but can maintain the data in JSON.

[srett]

After reading some tutorials I quickly got a SPARQL query going. If I didn't mess anything up the result is a bit underwhelming, half the phones from @mr-bobdobalina are missing, but still a total of ~1300 results.
It might take some time to massage the data a bit though:

• filter out models where the EXIF data is already the proper name (or close enough)
• model name always includes/starts with the manufacturer name, remove
• The property for the human readable manufacturer name is inconsistent, sometimes it's manufacturer, sometimes brand, sometimes developer
• Some entries have multiple brands and thus appear multiple times in the results

But it should be possible to use this for quickly extending the lookup table. I guess I've got something fun to ponder about this weekend. :-)

[srett]

Not 100% sure, but I remember to have added some code that automatically detects and fixes this. If not, it might be worth implementing.

Found the code, it just fails here since "NIKON CORPORATION" is not a prefix of "NIKON D80", but that should be fixed once the normalization is in place.

A few technical questions have popped up so far: As of now, the normalization/translation of the exif make and model names happens before inserting into the camera table. That means that every time the lookup tables in camera_makes.go get extended/updated, the db table has to be wiped and regenerated by looking at every photo's EXIF data again. Otherwise, we'd end up with old untranslated entries next to the new ones as soon as the user uploads new photos from that camera. That could be quite expensive on large installations. While this is not the end of the world, it's something to keep in mind for future updates.
Would it make sense to change this while at it?
I think it should be as simple as using the raw exif make/model as input for the CameraSlug, not the corrected one. This way, it would be enough to clear and rebuild the human readable columns on each update, which would be very fast since we don't need to scan all photos again, just one sample photo for each camera. Another possibility would be to have two more columns, so we store make and model two times, each in the raw and corrected form in that table. That saves that weird "hey we have to scan one photo for each camera model if the lookup tables changed" dance.

The other problem I found are devices where the exif make field is either wrong, or contains a company name that isn't shown anywhere on the product, for example "MADV" "QJXJ01FJ" -> "Xiaomi Mi Sphere", or "Google" "ONEPLUS A6013" -> OnePlus 6T. We cannot simply add an entry to the manufacturer lookup table to map "Google" to "OnePlus", since then all the Pixel devices would suddenly have OnePlus as the maker. Also, while it doesn't seem to be the case now, in the future we could have identical exif model names from different vendors, in which case the lookup by just model name is ambiguous.
Thus, my current plan is to have three lookup tables:

• New table: Manufacturer override: For the "wrong make" case. Lookup will be "exif make + exif model" -> "proper make", e.g. "Google|ONEPLUS A6013" -> "OnePlus"
• Make lookup: Same as the current one, just "exif make" -> "human readable make"
• Model lookup: Would change to "exif make + exif model" -> "human readable model", but logically still the same as before, e.g. "HUAWEI|CLT-L29" -> "P20 Pro", so in case any other company gives any of their phones or cameras an internal model name of "CLT-L29", we can handle that properly.

Looking forward to your thoughts and input.

[srett]

@lastzero just realized I filled in the wrong reply box, so I guess you didn't get a notification... 🙄

[lastzero]

I always get a notification 😂

[srett]

Hah ok, just making sure ;)

I kept working on it a bit and realized that the idea of having the original EXIF names in the cameras table creates all new problems:
Some phones use different exif model names for minor revisions, or different markets, but are still officially named the same, e.g. the Huawei P10 has 4 different EXIF models. So in the current version, where the lookup was made before writing to DB, just one DB entry was created for all 4 phones; but with my suggested change, we would get 4 DB entries, i.e. 4 cameras, even though they all have the same display name. So basically everything relating to cameras would be broken then. If the user filters by "Huawei P10", the query would have to to match all 4 camera IDs (assuming the user has photos from all 4 of them). 😮‍💨 Probably not worth it for now, needs more thinking...

Some more annoying things about the real world:

• Some companies changed name over time or merged with another one, but kept the old EXIF make for some time, e.g. "Konika Minolta" Kept using just "Minolta" in EXIF for a while, even though products were already sold under the "Konika Minolta" brand; so far it looks like defaulting to the old company name for those is the best choice.
• Some phones were released with different names in different markets

But at least I think I'm mostly done with the manufacturer lookup table, this is what I got so far:

{
    "acer": "Acer",
    "asus": "ASUS",
    "CASIO COMPUTER CO.,LTD": "Casio",
    "CASIO COMPUTER CO.,LTD.": "Casio",
    "EASTMAN KODAK COMPANY": "Kodak",
    "FUJIFILM": "Fujifilm",
    "Garmin-Asus": "Garmin",
    "Hewlett-Packard": "HP",
    "htc": "HTC",
    "HUAWEI": "Huawei",
    "KODAK": "Kodak",
    "KONICA MINOLTA": "Konica Minolta",
    "Leica Camera AG": "Leica",
    "LEICA": "Leica",
    "LG Electronics": "LG",
    "LGE": "LG",
    "lge": "LG",
    "Minolta Co., Ltd.": "Minolta",
    "MINOLTA CO.,LTD": "Minolta",
    "Motorol": "Motorola",
    "Motorola Mobility": "Motorola",
    "motorola": "Motorola",
    "NIKON CORPORATION": "Nikon",
    "NIKON": "Nikon",
    "OLYMPUS CORPORATION": "Olympus",
    "OLYMPUS DIGITAL CAMERA": "Olympus",
    "OLYMPUS IMAGING CORP.": "Olympus",
    "OLYMPUS OPTICAL CO.,LTD": "Olympus",
    "OLYMPUS": "Olympus",
    "PENTAX Corporation": "Pentax",
    "PENTAX": "Pentax",
    "Research In Motion": "BlackBerry",
    "RICOH IMAGING COMPANY, LTD.": "Ricoh",
    "RICOH": "Ricoh",
    "Samsung Electronics": "Samsung",
    "SAMSUNG TECHWIN Co.": "Samsung",
    "SAMSUNG TECHWIN": "Samsung",
    "Samsung Techwin": "Samsung",
    "samsung": "Samsung",
    "SAMSUNG": "Samsung",
    "SHARP": "Sharp",
    "SIGMA": "Sigma",
    "SONY": "Sony"
}

[lastzero]

@srett Thank you! I've added your suggestions (with minor changes) while working on the camera make and model normalization today. This was very helpful.

[srett]

@lastzero Ah, glad it was useful and sorry for never turning it into a proper PR. I was planning on adding go:generate stuff that would automatically update those files using wikidata and eventually got a bit overwhelmed as I'm not familiar with go and all the tooling/workflows involved.

[lastzero]

This has been released today: https://github.com/photoprism/photoprism/releases/tag/231011-63f708417

Any help in testing these changes is much appreciated!