Extras: Plex Dupefinder

desimaniac edited this page Nov 14, 2018 · 75 revisions
  1. Introduction
  2. Install
  3. Config
  4. Plex
  5. Run Plex Dupefinder

1. Introduction

Plex Dupefinder (by l3uddz) is a python app that finds duplicate versions of media (TV episodes and movies) in your Plex Library and tells Plex to remove the lowest quality versions (based on a scoring algorithm), either automatically or interactively (i.e. with a prompt on each find), leaving you with one high quality media file.

Note: For Mediabox/Feederbox setups, this can be install in either system.

The scoring is based on: non-configurable and configurable parameters.

  • Non-configurable parameters are: bitrate, duration, height, width, and audio channel.

  • Configurable parameters are: audio codec scores, video codec scores, video resolution scores, filename scores, and file sizes (can only be toggled on or off).

  • Note: bitrate, duration, height, width, audio channel, audio and video codecs, video resolutions (e.g. SD, 480p, 720p, 1080p, 4K, etc), and file sizes are all taken from the metadata Plex retrieves during media analysis.


Demo:

asciicast

2. Install

Run the following commands:

cd ~/cloudbox/
sudo ansible-playbook cloudbox.yml --tags plex_dupefinder

3. Config

Location

/opt/plex_dupefinder/config.json

Example

{
    "AUDIO_CODEC_SCORES": {
        "aac": 1000,
        "ac3": 1000,
        "dca": 2000,
        "dca-ma": 4000,
        "eac3": 1250,
        "flac": 2500,
        "mp2": 500,
        "mp3": 1000,
        "pcm": 2500,
        "truehd": 4500,
        "Unknown": 0,
        "wmapro": 200
    },
    "AUTO_DELETE": false,
    "FILENAME_SCORES": {
        "*.avi": -1000,
        "*.ts": -1000,
        "*.vob": -5000,
        "*1080p*BluRay*": 15000,
        "*720p*BluRay*": 10000,
        "*dvd*": -1000,
        "*HDTV*": -1000,
        "*PROPER*": 1500,
        "*Remux*": 20000,
        "*REPACK*": 1500,
        "*WEB*CasStudio*": 5000,
        "*WEB*KINGS*": 5000,
        "*WEB*NTB*": 5000,
        "*WEB*QOQ*": 5000,
        "*WEB*SiGMA*": 5000,
        "*WEB*TBS*": -1000,
        "*WEB*TROLLHD*": 2500,
        "*WEB*VISUM*": 5000
    },
    "PLEX_SECTIONS": {
        "Movies": 1,
        "TV": 2
    },
    "PLEX_SERVER": "https://plex.yourdomain.com",
    "PLEX_TOKEN": "",
    "SCORE_FILESIZE": true,
    "SKIP_LIST": [],
    "VIDEO_CODEC_SCORES": {
        "h264": 10000,
        "h265": 5000,
        "hevc": 5000,
        "mpeg1video": 250,
        "mpeg2video": 250,
        "mpeg4": 500,
        "msmpeg4": 100,
        "msmpeg4v2": 100,
        "msmpeg4v3": 100,
        "Unknown": 0,
        "vc1": 3000,
        "vp9": 1000,
        "wmv2": 250,
        "wmv3": 250
    },
    "VIDEO_RESOLUTION_SCORES": {
        "480": 3000,
        "720": 5000,
        "1080": 10000,
        "4k": 20000,
        "sd": 1000,
        "Unknown": 0
    }
}

Details

Audio Codec Scores

  • You can set AUDIO_CODEC_SCORES to your preference.

  • The default settings should be sufficient for most.

Auto Delete

  • Under AUTO_DELETE, set your desired option.

    • "AUTO_DELETE": true, - Plex Dupefinder will run in automatic mode.

    • "AUTO_DELETE": false, - Plex Dupefinder will run in interactive mode. (Default)

      • Options:

        • Skip (i.e. keep both): 0

        • Choose the best one (and delete the rest): b

        • Select the item to keep (and delete the rest): # (i.e. 1, 2, 3, etc).

Filename Scores

  • You can set FILENAME_SCORES to your preference.

  • The default settings should be sufficient for most.

Plex Sections

  1. Go to Plex and get all the names of your Plex Libraries you want to find duplicates in.

    • Example Library:

  2. Under PLEX_SECTIONS, type in the Plex Library Name (exactly) and specify the Library Type (not the Section ID): 1 for movies or 2 for TV shows.

    • Format:

      "PLEX_SECTIONS": {
        "LIBRARY_NAME_1": #,
        "LIBRARY_NAME_2": #
      },
      

      # = Library Type: 1 or 2

    • For basic libraries, this will look like:

      "PLEX_SECTIONS": {
        "Movies": 1,
        "TV": 2
      },
    • For more advanced libraries (e.g. Customizing Plex Libraries), this will look like:

      "PLEX_SECTIONS": {
         "3D": 1,
         "4K": 1,
         "Foreign": 1,
         "Hollywood": 1,
         "Kids": 1,
         "TV": 2
      },

Plex Server URL

  • Pre-filled with your Plex server's URL.

Plex Token

  1. Obtain a Plex Access Token: See Plex Access Token.

  2. Add the Plex Access Token to "PLEX_TOKEN" so that it now appears as "PLEX_TOKEN": "xxxxxxxxxxxxxx",.

    • Note: Make sure it is within the quotes (") and there is a comma (,) after it.

Filesize Scores

  • "SCORE_FILESIZE": true will add more points to the overall score based on the actual file size. ]

  • Note: In some situations (e.g. a bad encode resulting in a large size), this may be something you want to turn it off (false). However, the default settings (i.e. true) should be sufficient for most.

  • The default settings should be sufficient for most.

Skip List

  • In Auto Delete mode, any file paths matching the patterns (i.e folders), listed in SKIP_LIST, will be ignored.

  • Example:

    "SKIP_LIST": ["/Movies4K/"]
  • The default settings should be sufficient for most.

Video Codec Scores

  • You can set VIDEO_CODEC_SCORES to your preference.

  • The default settings should be sufficient for most.

Video Resolution Scoring

  • You can set VIDEO_RESOLUTION_SCORES to your preference.

  • The default settings should be sufficient for most.

4. Plex

You will need to make sure that "Allow media deletion" is enabled in Plex.

  1. In Plex, click the Settings icon (top right) -> "Server" (top) -> "Library" (left).

  2. Set the following:

    • "Allow media deletion": enabled
  3. Click "SAVE CHANGES".

5. Run Plex Dupefinder

  • Simply run the following command:

    plexdupes
    

Home

Basics

  1. Introduction
  2. Cloudbox Install Types
  3. Cloudbox Paths
  4. Accessing Cloudbox Apps

Prerequisites

  1. Presumptions
  2. Server
  3. Domain Name
  4. Cloudflare
  5. Cloud Storage
  6. Plex / Emby - Account
  7. Usenet vs. BitTorrent

Install

Cloudbox
  1. User Account
  2. Dependencies
  3. Ansible Vault
  4. Settings
  5. Cloudbox
  6. Application Setup
    1. Rclone
    2. Plexdrive
    3. NZBGet
    4. ruTorrent
    5. NZBHydra2
    6. Jackett
    7. Plex Media Server
    8. Plex Autoscan
    9. Sonarr
    10. Radarr
    11. Lidarr
    12. PlexPy (Tautulli)
    13. Ombi
    14. Portainer
    15. Organizr

Mediabox
  1. User Account
  2. Dependencies
  3. Ansible Vault
  4. Settings
  5. Mediabox
  6. Application Setup
    1. Rclone
    2. Feeder Mount
    3. Plexdrive
    4. Plex Media Server
    5. Plex Autoscan
    6. PlexPy (Tautulli)
    7. Ombi

Feederbox
  1. User Account
  2. Dependencies
  3. Ansible Vault
  4. Settings
  5. Feederbox
  6. Application Setup
    1. Rclone
    2. Plexdrive
    3. NZBGet
    4. ruTorrent
    5. NZBHydra2
    6. Jackett
    7. Sonarr
    8. Radarr
    9. Lidarr
    10. Portainer
    11. Organizr

Recommended Reading

More Information

Advanced Configuration

Experimental

Extras

Docs coming soon for ...

  • Radarr4K
  • Sonarr4K

Misc Guides

Reference

Troubleshooting

Links

Community Project

External Links

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.