Skip to content

The PG EcoSystem

Admin9705 edited this page Feb 22, 2019 · 21 revisions

CLICKING the STAR in the UPPER RIGHT promotes our PROJECT on GITHUB! - WARNING, PLEASE VISIT https://pgblitz.com/wikis/ - This is the old wiki!


There's a ton of moving parts involved in a automated media server system. You can think of Plexguide as an easy-to-assemble lego set with all the pieces already sourced in the box. Many of the more complex pieces are already packaged and configured to work together with the help of Docker and Ansible.

Bitorrent, Usenet, Indexers and Trackers

The journey begins on the high seas of BitTorrent and Usenet. To sail these waters, one requires the guidance of Torrent Tracker sites and Usenet Indexers. You can think of these as a digital treasure map (but more like a phonebook directory). They help you find content for Plex by pointing out the location of the files you request. Trackers and indexers can be public, semi-public (as in pay-to-play) and private (invite required). Private trackers and indexers will get net the best content, the fastest download speeds and the best selection. Note that if you go with Torrents, you'll need a bigger hard drive to seed files. Also note there is a special place in hell for people who don't seed. If you go with Usenet, know that many older files (and many new ones) will be blown out of the water by the DMCA Royal Navy before you can download the file, so gaining access to private indexers or having multiple indexers and block accounts will be necessary. You can either pick using torrents or usenet, but I recommend both— Torrents for older and rare files, and Usenet for fast access to recent files.

NZBHydra & Jackett

These are helper programs that aggregate all your Usenet indexers (NZBhydra) and Torrent trackers (Jackett) together in one place. This is optional, as Radarr/Sonarr/Lidarr can (usually) be configured to access your indexers and torrent trackers directly. If you have many indexers/trackers or private ones, I recommend going with this route.

These programs are optional and many features available here are already packed into Radarr and Sonarr.

Media Managers (Radarr, Sonarr & Lidarr)

Radarr is for Movies, Sonarr is for TV and Lidarr is for Music. These programs are your vigilant crewmates on lookout for new movies, shows and albums. They automatically check the RSS feeds from your indexers and trackers (directly, or indirectly via nzbhydra/jackett) and tell NZBget/Deluge to snatch the file immediately if it matches what you request it to. They can also search for older releases. You can configure these to look for content based on lists, so you don't have to import content manually. Note that these programs can also see what's in UnionFS (a filesystem) and will not download media you've already plundered). More on that later.

Lidarr is still in the alpha stage of devlopment, so it may contain bugs!

Download Clients (NZBget & Deluge)

NZBget handles Usenet downloads, Deluge handles torrent downloads. They assemble the file together in temporary directories, perform a integrity check based on file hashes and repairs them if necessary.

If you are using public trackers, a VPN is highly recommended.

Post Processing

Once the files are finished downloading, they are moved (or copied for torrent files) into "completed" directories, which Radarr/Sonarr/Lidarr then processes by stripping out .nfo's, samples and extras and renaming the file in a way that Plex can find the correct metadata. After processing, the files are then packaged into an appropriately named folder and placed onto a loading dock (/mnt/move*) where it awaits shipment to Google Drive

You can easily upload a pre-existing library on local disk (or network mount) to google drive by manually moving the file to /mnt/move/Movies.

RClone & move.sh

Rclone is a utility program that syncs files to the cloud. You can think of rclone as a truck that arrives every 8 minutes, driven by a systemd daemon called move.sh that delivers your files to the Google Drive warehouse for long-term storage. The workers at Google don't exactly have the best reputation when it comes to respecting your privacy, so you may wish to obfuscate your booty (get it?) by encrypting your files before it's whisked away to the great cloud in the sky.

PlexDrive

Google doesn't like it when you spam it with requests every time Plex issues a library rescan, and will issue a 24 hour API ban if you hit their servers too hard. To mitigate this issue, you'll need to mount your Google Drive with PlexDrive. PlexDrive caches the metadata of your gdrive (filenames, locations) into a database, then presents that information as a pseudo directory called a fuse mount. Now when Plex scans this directory, it only issues new api requests to google for files that have been recently changed, instead of every single file of your entire library.

UnionFS

There's a caveat though, Plex doesn't actually scan the pseudo-directory. Plexdrive is actually mounted onto a union filesystem called UnionFS which combines the contents of what's in the loading dock awaiting upload (/mnt/move/) and the contents of PlexDrive into one cohesive directory (/mnt/unionfs). Note that Radarr/Sonarr/Lidarr can read from this directory so it won't download duplicates. The advantage to having UnionFS combine these two directories is the ability to watch files right after they are done downloading, instead of waiting for it to be uploaded.

Plex Media Server

From there, Plex scans the library from UnionFS where it's indexed and presented with metadata like IMDB ratings, movie posters and descriptions in a presentable format. By default, Plex works on port 32400, not port 80/443 like most websites.

Traefik Reverse Proxy

That's an ugly port number though, so we'll present it in a nicer way on port 443 (https) with your own domain name with Traefik. Traefik is a reverse proxy that routes traffic from various TCP ports into port 443 and applies SSL encryption so traffic can't be sniffed en route to your users. Note that all the other programs also have their traffic routed through Traefik, I just haven't shown it on the diagram for simplicity's sake.

Ombi

Finally, users can then browse to your website and watch. If they'd like to make a request for another movie or show, they can access Ombi, which is basically a front-end middleman that passes on media requests to Radarr or Sonarr. The advantage is that you won't have to give your users admin access to the rest of your configs and you can deny/allow requests if you want. It can also send all your users an email with what shows have been recently added to your Plex library.


Visit our Website By Click the Picture Below! N|Demo

N|Solid N|Solid

http://PGBlitz.wiki

Useful Starter Links

  1. Introduction
  2. Blitz YouTube Channel
  3. G-Suite Signup
  4. Recommended NewsHosts

Prior Planning 101

  1. Recommended Pre-Reading
  2. Server - Storage Planning
  3. Usenet or BitTorrent
  4. PG Editions
  5. PGBlitz Repos
  6. Common Issues

Deploy & Config

PGBlitz
  1. SSH Server Access
  2. Create a SUDO User
  3. Install PGBlitz
  4. WatchTower
  5. Remote Path Mappings

Data Transport
  1. PG Clone
    1. Google OAuth Keys
    2. PG Move
    3. PG Blitz
    4. 2nd HD Option

Key Components

  1. Traefik
  2. Port Guard
  3. PG Shield
  4. PG Press
  5. Google Cloud - GCE
    1. Automated setup
  6. Hetzner Cloud
  7. PG Fork
  8. Extra PG Commands

Blitz App Info

  1. Accessing PG Apps
  2. App Port Scheme
  3. Plex to AutoScan

Tools & Services

Core Apps [Expand]
  1. Emby
  2. Jackett
  3. Jellyfin
  4. LazyLibrarian
  5. Lidarr
  6. Netdata
  7. NZBGet
  8. Ombi
  9. Plex
  10. Portainer
  11. qBittorrent
  12. Radarr
  13. RuTorrent
  14. SABNZBD
  15. Sonarr
  16. Tautulli
Community Apps [Expand]
  1. AirSonic
  2. AllTube
  3. Bazarr
  4. Beets
  5. Bitwarden
  6. BookSonic
  7. cAdvisor
  8. Cloud Commander
  9. DDClient
  10. Deezloaderremix
  11. Deluge
  12. DelugeVPN
  13. Duplicati
  14. EmbyStats
  15. FlexGet
  16. Gazee
  17. Gitea
  18. Handbrake
  19. Headphones
  20. Heimdall
  21. HomeAssistant
  22. HTPCManager
  23. JDownloader2
  24. jd2-openvpn
  25. Kitana
  26. Logarr
  27. MakeMKV
  28. MariaDB
  29. McMyadmin
  30. MEDUSA
  31. Mellow
  32. Monitorr
  33. Muximux
  34. Mylar
  35. NextCloud
  36. NowShowing
  37. NZBHydra2
  38. NZBThrottle
  39. Organizr
  40. pyLoad
  41. Radarr4k
  42. Resilio
  43. rflood-openvpn
  44. rutorrent-openvpn
  45. ShareSite
  46. Shoko Anime Server
  47. SpeedTest
  48. SyncLounge
  49. Syncthing
  50. The Lounge
  51. Transmission
  52. Trakt.or
  53. Ubooquity
  54. x TeVe
  55. Zammad

Misc Info & Articles

  1. Change Server Time
  2. Fail2Ban
  3. Custom Containers
Scripts
  1. Scripting Area
    1. Auto Upload for Sonarr and Lidarr
    2. Auto Delete Log
    3. Plex Plugin DB Backup
    4. Plex fast BACKUP
Clone this wiki locally
You can’t perform that action at this time.