Google Photos Deduper

Locally run web app + Chrome extension to delete duplicates in Google Photos. Built with:

Demo

Getting Started

While a hosted web app would be ideal, one is not currently provided due to API usage limits, the overhead of Google's app verification process, cost, and user privacy considerations. Instead, follow these instructions to get the app up and running locally:

Setup

1. Install Docker Desktop on your system.

2. Clone this repository.

3. Create a Google Cloud project and OAuth credentials.

• Create a Google Cloud project (Guide)
  • Project name: Enter Photos Deduper
  • Select the project
• Go to APIs & Services > Enable APIs and Services
  • Search for Photos Library API
  • Enable
• Go to APIs & Services > OAuth consent screen
  • User Type: Choose External
  • Create
    • App name: Enter Photos Deduper
    • User support email: Choose your email
    • Developer contact information: Enter your email
    • Save and Continue
  • Add or remove scopes:
    • Manually add scopes:
      • https://www.googleapis.com/auth/userinfo.profile
      • https://www.googleapis.com/auth/userinfo.email
      • https://www.googleapis.com/auth/photoslibrary
    • Update
    • Save and Continue
  • Test users:
    • Add your email (and any others you want to use the tool with)
    • Save and Continue
• Go to APIs & Services > Credentials > Create Credentials > OAuth client ID
  • Application type: Choose Web application
  • Name: Enter Photos Deduper Web Client
  • Authorized JavaScript origins: Enter http://localhost
  • Authorized redirect URIs: Enter http://localhost/auth/google/callback
  • Create
• Download the JSON file

4. Set up local environment variables.

• cp example.env .env
• Generate FLASK_SECRET_KEY with python -c 'import secrets; print(secrets.token_hex())' and add it to .env.
• Add GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET from the client_id and client_secret values from the client secret file created above.

Start

1. Run docker-compose up from the project directory.
2. Load http://localhost and follow the instructions from there!
   • Install the Chrome Extension once you want to delete duplicates.

Support

If you found a bug or have a feature request, please open an issue.

If you have questions about the tool, please post on the discussions page.

Development

• Python app
  • Flask is set to debug mode, so live reloading is enabled.
  • Debugging with debugpy is supported. See launch.json.
• React app
  • Utilizes Vite for HMR and building.
• Chrome extension
  • Utilizes the CRXJS Vite Plugin for HMR and building.

Motivation

I've been a long-time user of Google Photos. When Picasa Web Albums retired, my cloud photos and albums moved to Google Photos. I have used nearly every desktop client Google provided from Picasa, to the old Google Photos desktop uploader, to Google Drive's built-in Photos integration, and finally to Backup and Sync.

Google has improved duplicate detection upon upload in recent years, but that wasn't always the case. I have tens of thousands of photos across hundreds of albums that were at some point duplicated by a desktop client. Also, even today, deleting, re-uploading, then restoring a photo results in a duplicate.

This could probably be solved by clearing out my Photos data and re-uploading everything. However, that would remove all album organization and photo descriptions. Instead, it's preferred to remove duplicates in-place. Searches show interest in this feature from the Google Photos user base, but it hasn't ever made its way into the product.

The existing tools I could find for this problem did so only with media on the local computer, felt scammy, or didn't fully automate the deletion process. So I created this one.

It turns out the Google Photos API is quite limited. While apps can read limited metadata about the media items in a user's library, they cannot delete media items (photos and videos), and they can only modify media items uploaded by the app itself. This means we can't, for example, add all of the duplicates to an album for the user to review. This necessitates some kind of tool to automate the deletion of duplicates. Since we've already bought in to the Google ecosystem as a Photos user, I chose to do this with a complementary Chrome extension.

Say Thanks

If you found this project useful, give it a star!