Some downloaded files are duplicates

[gilesknap]

In some cases, photos with name clashes seem to be downloading more than once and replacing some of their name clashes.
e.g.
Filename 'Picture 045.jpg':-

• Shows 16 entries in the Db with Unique RemoteIds
• Shows 16 unique images on Google Photos Web with search "Picture 045.jpg"
• Shows 15 images in the gphotos-sync download but there are 3 duplicate pairs and 5 missing from above

[gilesknap]

Looks like it is a duplicate number issue.
sqlite> select DuplicateNo, createdate, FileName, description from syncfiles where FileName like 'Picture 045%' order by Filename;
0|2002-06-21 22:35:26|Picture 045 (2).jpg|Y2002 M06 D22 Gatecrasher
1|2002-06-21 22:35:26|Picture 045 (2).jpg|Y2002 M06 D03 Neil&Fran
1|2002-05-08 13:25:43|Picture 045 (2).jpg|Y2002 M05 D09 Cycle Minolta Test
1|2003-04-10 22:06:57|Picture 045 (2).jpg|Y2003 M04 D10 Tracey and Cullen Birthday
1|2002-08-04 04:05:14|Picture 045 (2).jpg|Y2002 M08 D03 Music in the Air_DCF 1.0
0|2002-08-31 18:40:53|Picture 045 (3).jpg|Y2002 M08 D31 Heath n Laura
0|2002-05-30 13:10:29|Picture 045 (3).jpg|Y2002 M05 D31 NTC Lunch
0|2002-05-10 17:13:16|Picture 045 (4).jpg|Y2002 M05 D10 Weekend London
0|2002-05-16 13:55:35|Picture 045 (5).jpg|Y2002 M05 D18 Palmer Anniversary
0|2002-05-31 15:51:58|Picture 045 (6).jpg|Y2002 M06 D01 Henley Lunch
0|2002-05-31 14:52:12|Picture 045-ANIMATION.gif|
0|2002-05-08 13:25:43|Picture 045.jpg|Y2002 M05 D24 Loche Fine Restaurant
0|2002-10-26 13:14:03|Picture 045.jpg|Y2002 M10 D26 London Aquarium
0|2002-06-02 22:24:34|Picture 045.jpg|Y2002 M06 D09 Giles Birthday
0|2003-04-10 22:06:57|Picture 045.jpg|Y2003 M04 D18 Easter Sainte Foy - Giles
0|2002-08-04 04:05:14|Picture 045.jpg|Y2002 M08 D10 Ibiza
0|2005-10-17 09:58:35|Picture 045.jpg|Y2005 M10 D15 Australia - Neil Ixus

It looks like the duplicate number 1 files are the issue?

[gilesknap]

Aha. Because there was already a file with ORIGINAL filename "Picture 045 (2).jpg". When two files with original name "Picture 045.jpg" were seen, the second was given a (2) suffix and clashed with the pre-existing (2) suffixed file.

I think the neatest thing to do would be to strip any (nnn) suffixes from original filenames - but this is lying in the metadata.
Or use a different suffix scheme

[gilesknap]

Fixed by stripping (nnn) from original file names. Makes for neater names anyway.

Now the only duplicates that show up on a 'self-comparison' are genuinely in the library twice and have a UID to demonstrate their