Extras: Plex DupeFinder
Plex DupeFinder (by l3uddz) is a python app that finds duplicate versions of media (TV episodes and movies) in your Plex Library and tells Plex to remove the lowest quality versions (based on a scoring algorithm), either automatically or interactively (i.e. with a prompt on each find), leaving you with one high quality media file.
Note: For Mediabox/Feederbox setups, this can be install in either system.
The scoring is based on: non-configurable and configurable parameters.
-
Non-configurable parameters are: bitrate, duration, height, width, and audio channel.
-
Configurable parameters are: audio codec scores, video codec scores, video resolution scores, filename scores, and file sizes (can only be toggled on or off).
-
Note: bitrate, duration, height, width, audio channel, audio and video codecs, video resolutions (e.g. SD, 480p, 720p, 1080p, 4K, etc), and file sizes are all taken from the metadata Plex retrieves during media analysis.
Demo:
Run the following commands:
cd ~/cloudbox/
sudo ansible-playbook cloudbox.yml --tags plex_dupefinder
/opt/plex_dupefinder/config.json
{
"AUDIO_CODEC_SCORES": {
"aac": 1000,
"ac3": 1000,
"dca": 2000,
"dca-ma": 4000,
"eac3": 1250,
"flac": 2500,
"mp2": 500,
"mp3": 1000,
"pcm": 2500,
"truehd": 4500,
"Unknown": 0,
"wmapro": 200
},
"AUTO_DELETE": false,
"FIND_DUPLICATE_FILEPATHS_ONLY": false,
"FILENAME_SCORES": {
"*.avi": -1000,
"*.ts": -1000,
"*.vob": -5000,
"*1080p*BluRay*": 15000,
"*720p*BluRay*": 10000,
"*dvd*": -1000,
"*HDTV*": -1000,
"*PROPER*": 1500,
"*Remux*": 20000,
"*REPACK*": 1500,
"*WEB*CasStudio*": 5000,
"*WEB*KINGS*": 5000,
"*WEB*NTB*": 5000,
"*WEB*QOQ*": 5000,
"*WEB*SiGMA*": 5000,
"*WEB*TBS*": -1000,
"*WEB*TROLLHD*": 2500,
"*WEB*VISUM*": 5000
},
"PLEX_LIBRARIES": [
"Movies",
"TV"
],
"PLEX_SERVER": "https://plex.yourdomain.com",
"PLEX_TOKEN": "",
"SCORE_FILESIZE": true,
"SKIP_LIST": [],
"VIDEO_CODEC_SCORES": {
"h264": 10000,
"h265": 5000,
"hevc": 5000,
"mpeg1video": 250,
"mpeg2video": 250,
"mpeg4": 500,
"msmpeg4": 100,
"msmpeg4v2": 100,
"msmpeg4v3": 100,
"Unknown": 0,
"vc1": 3000,
"vp9": 1000,
"wmv2": 250,
"wmv3": 250
},
"VIDEO_RESOLUTION_SCORES": {
"480": 3000,
"720": 5000,
"1080": 10000,
"4k": 20000,
"sd": 1000,
"Unknown": 0
}
}
-
You can set
AUDIO_CODEC_SCORES
to your preference. -
The default settings should be sufficient for most.
-
Under
AUTO_DELETE
, set your desired option.-
"AUTO_DELETE": true,
- Plex DupeFinder will run in automatic mode. -
"AUTO_DELETE": false,
- Plex DupeFinder will run in interactive mode. (Default)-
Options:
-
Skip (i.e. keep both):
0
-
Choose the best one (and delete the rest):
b
-
Select the item to keep (and delete the rest):
#
(i.e.1
,2
,3
, etc).
-
-
-
-
Finds duplicates that only share the same file path.
"FIND_DUPLICATE_FILEPATHS_ONLY": false,
-
This option has a very limited use case, i.e. in instances where Plex may have glitched and created multiple duplicates of the same media item.
-
If using this setting, we recommend using UnionFS-Fuse that can generate whiteout files (
*_HIDDEN~
) to prevent the deletion of the actual file on the system. The_HIDDEN~
files can then be removed afterwards or even during the dupe cleanup (e.g.watch -n 5 rm -rf /mnt/local/.unionfs-fuse/*
). -
The default settings should be sufficient for most.
-
You can set
FILENAME_SCORES
to your preference. -
The default settings should be sufficient for most.
-
Go to Plex and get all the names of your Plex Libraries you want to find duplicates in.
-
Example Library:
-
-
Under
PLEX_LIBRARIES
, add in a list of Plex libraries that you want scanned for duplicates.-
Format:
"PLEX_LIBRARIES": [ "LIBRARY_NAME_1", "LIBRARY_NAME_2" ],
-
Example:
"PLEX_LIBRARIES": [ "Movies", "TV" ],
-
- Pre-filled with your Plex server's URL.
-
Obtain a Plex Access Token: See Plex Access Token.
-
Add the Plex Access Token to
"PLEX_TOKEN"
so that it now appears as"PLEX_TOKEN": "xxxxxxxxxxxxxx",
.- Note: Make sure it is within the quotes (
"
) and there is a comma (,
) after it.
- Note: Make sure it is within the quotes (
-
"SCORE_FILESIZE": true
will add more points to the overall score based on the actual file size. -
Note: In some situations (e.g. a bad encode resulting in a large size), this may be something you want to turn it off (
false
). However, the default settings (i.e.true
) should be sufficient for most. -
The default settings should be sufficient for most.
-
In Auto Delete mode, any file paths matching the patterns (i.e folders), listed in
SKIP_LIST
, will be ignored. -
Example:
"SKIP_LIST": ["/Movies4K/"]
-
The default settings should be sufficient for most.
-
You can set
VIDEO_CODEC_SCORES
to your preference. -
The default settings should be sufficient for most.
-
You can set
VIDEO_RESOLUTION_SCORES
to your preference. -
The default settings should be sufficient for most.
You will need to make sure that "Allow media deletion" is enabled in Plex.
-
In Plex, click the Settings icon (top right) -> "Server" (top) -> "Library" (left).
-
Set the following:
- "Allow media deletion":
enabled
- "Allow media deletion":
-
Click "SAVE CHANGES".
-
Simply run the following command:
plex_dupefinder
- Overview
- Presumptions
- Server
- Domain Name
- Cloudflare
- Cloud Storage
- Plex / Emby - Account
- Usenet vs. BitTorrent
Cloudbox
- Overview
- Dependencies (Choose only one of these)
- Settings
- Preinstall (Choose only one of these)
- SSH
- Ansible Vault
- Rclone
- Cloudbox (Choose only one of these)
- Application Setup
- Next Steps
Feederbox (do this first)
- Overview
- Dependencies
- Settings
- Preinstall
- SSH
- Ansible Vault
- Rclone
- Feederbox (Choose only one of these)
- Application Setup
- Next Steps
Mediabox
- Overview
- Dependencies
- Settings
- Preinstall
- SSH
- Ansible Vault
- Rclone
- Mediabox (Choose only one of these)
- Application Setup
- Next Steps
- Cloudplow (Media Uploader)
- cb utility script (Develop branch only}
- Updating Cloudbox (Choose only one of these)
- Updating Cloudbox Apps
- Removing Cloudbox Apps
- Resetting Cloudbox Apps
- Migrating Cloudbox
- Settings Updater
- Ansible Vault Primer
- Plex Access Token
- Plex Autoscan Extras
- Pushover
- Google Drive API Client ID and Client Secret
- Useful Docker Commands
- Add Your Own Docker Container into Cloudbox
- Revoking SSL Certificates
- Feeder Mount
- Adding a Subdomain
- HTTP Auth Support
- Emby
- Nextcloud
- Resilio Sync
- Plex DupeFinder
- Heimdall
- NZBHydra v1
- Plex Requests
- Sickbeard MP4 Automator
- SABnzbd
- Traktarr
See Community Wiki.