Skip to content

PG Blitz

Clownfused edited this page Apr 17, 2019 · 27 revisions
WANT TO HELP? CLICK THE ★ (STAR LOGO) in the Upper-Right!

  • 📂 [Click Here] - Google G-Suite (Unlimited Hard Drive Space & Storage)
  • 📂 [Click Here] - Top Performance NewsHost! - Blitz Members Receive a 58% Discount

Reference Shortcut - http://pgblitz.pgblitz.com | Discord ( !pgblitz )




Table of Contents

  1. Intro
  2. PG Blitz Advantages & Disadvantages
  3. PG Blitz - UnEncrypted
  4. PG Blitz - Encrypted Setup
  5. How it works

1. Intro

PG Blitz was based on Flicker-Rate's SuperTransfer concept. SuperTransfer worked, but was complicated and created many problems. Over the course of the Summer in 2017, I was thinking of the word blitz (in about storming a battle field) and was just thinking about packets of data just collecting in a spot and attacking the first thing they see (the GDSA keys). To help streamline the concept after creating the code, Physik streamlined it further to ensure that data integrity was maintained. As a result, we now have PG Blitz, including the encrypted version. PG Blitz has been further simplified and resulted in the creation of PG Clone; which keeps executes deployments in a streamlined manner.

2. PG Blitz Advantages - Disadvantages

Advantages of PG Blitz over PG Move:

  • Breaks past 750GB daily upload barrier
  • Works great for GCE Edition

Disadvantages of PG Blitz over PG Move:

  • More complicated to set up
  • No Throttle Controls (point is fast upload)
  • Uses TeamDrives over GDrive (still works, but more stuff to add)

3. PG Blitz - Unencrypted Setup

i. PG YouTube - PG Blitz Unencrypted

ii. Information

The setup for this process is easy.

  • Have the Google OAuth Keys generated and saved!
  • Select PG Blitz Unencrypted
  • Goto PG Clone - OAuth & Mounts
  • Set the Client ID
  • Set the Secret ID
  • Set your team drive, validate, and select the correct one
  • Select GDrive under the rclone configuration and validate the link
  • Select TDrive under the rclone configuration and validate the link
  • Exit to the main menu
  • Select Deploy

4. PG Blitz - Encrypted Setup

i. PG YouTube - PG Blitz Encrypted

ii. Information

The setup for this process is easy.

  • Have the Google OAuth Keys generated and saved!
  • Select PG Blitz Encrypted
  • Goto PG Clone - OAuth & Mounts
  • Set the Client ID
  • Set the Secret ID
  • Set your team drive, validate, and select the correct one
  • Set a Primary Password & SALT Password
  • Select GDrive under the rclone configuration and validate the link
  • Select TDrive under the rclone configuration and validate the link
  • Exit out and then go to Key Generation
  • Login to the Project
  • Create or Establish a prior project
  • Create the Keys
  • (Optional) Backup the keys so, in the future, you can just recover them
  • Exit to the main menu
  • Select Deploy

5. How it works

  1. When radarr/lidarr/sonarr imports your downloads from /mnt/downloads/_downloader_/_category_/... it reads the download mappings and instead of looking for /mnt/downloads/_downloader_/_category_/... it looks for /mnt/unionfs/_downloader_/_category_/.... So at this point, it's accessing the files inside the merger.
  2. Now Arr will do a hardlink/mv operation to rename and organize the media. Arr will think it's putting the files in /mnt/unionfs/_category_/..., but it's actually putting them into /mnt/downloads/_category_/...
  3. At this point your files are imported and renamed, they are accessible at /mnt/unionfs/_category_/...
  4. The blitz or move script runs and ignores any "known downloader" folders inside /mnt/downloads, and moves everything it doesn't ignore (like /mnt/downloads/_category_) to /mnt/move for uploading.
  5. Now your files are inside /mnt/move for uploading, run pgblitz to see the status.

-- Arr has to organize the files, To hardlink inside mergerfs, it requires everything being inside the merger (aka /mnt/unionfs) for rapid importing to work. The downloader mappers tell arr to look in /mnt/unionfs/nzbget so then it does a mv. But underlying, mergerfs will do a rename in the real location, which is /mnt/downloads. There isn't a way to force that rename operation to occur in /mnt/move instead or we would just do that, this is a behavior we cannot change.

Files that exist in /mnt/downloads/_category_ will be moved for uploading.

Blitz runs in a batch, it grabs everything ready at that time and uploads it. It will not move other files in the background while actively uploading!

Arr however will continue to import and organize the next set of files. So typically files sit in /mnt/downloads/_category_ for 0-45m (depending on active upload).

During this entire process, once arr does the import, the file stays at /mnt/unionfs/_category_/[...] this is so plex can scan it and have it available without first waiting for it to be uploaded.

📂 Click Here - NZB's from USENET Server - PG Members Receive a 60% Discount

http://PGBlitz.wiki

Useful Starter Links

  1. Introduction
  2. Blitz YouTube Channel
  3. G-Suite Signup
  4. Recommended NewsHosts

Prior Planning 101

  1. Recommended Pre-Reading
  2. Server - Storage Planning
  3. Usenet or BitTorrent
  4. PG Editions
  5. PGBlitz Repos
  6. Common Issues

Deploy & Config

PGBlitz
  1. SSH Server Access
  2. Create a SUDO User
  3. Install PGBlitz
  4. WatchTower
  5. Remote Path Mappings

Data Transport
  1. PG Clone
    1. Google OAuth Keys
    2. PG Move
    3. PG Blitz
    4. 2nd HD Option

Key Components

  1. Traefik
  2. Port Guard
  3. PG Shield
  4. PG Press
  5. Google Cloud - GCE
    1. Automated setup
  6. Hetzner Cloud
  7. PG Fork
  8. Extra PG Commands

Blitz App Info

  1. Accessing PG Apps
  2. App Port Scheme
  3. Plex to AutoScan

Tools & Services

Core Apps [Expand]
  1. Emby
  2. Jackett
  3. Jellyfin
  4. LazyLibrarian
  5. Lidarr
  6. Netdata
  7. NZBGet
  8. Ombi
  9. Plex
  10. Portainer
  11. qBittorrent
  12. Radarr
  13. RuTorrent
  14. SABNZBD
  15. Sonarr
  16. Tautulli
Community Apps [Expand]
  1. AirSonic
  2. AllTube
  3. Bazarr
  4. Beets
  5. Bitwarden
  6. BookSonic
  7. cAdvisor
  8. Cloud Commander
  9. DDClient
  10. Deezloaderremix
  11. Deluge
  12. DelugeVPN
  13. Duplicati
  14. EmbyStats
  15. FlexGet
  16. Gazee
  17. Gitea
  18. Handbrake
  19. Headphones
  20. Heimdall
  21. HomeAssistant
  22. HTPCManager
  23. JDownloader2
  24. jd2-openvpn
  25. Kitana
  26. Logarr
  27. MakeMKV
  28. MariaDB
  29. McMyadmin
  30. MEDUSA
  31. Mellow
  32. Monitorr
  33. Muximux
  34. Mylar
  35. NextCloud
  36. NowShowing
  37. NZBHydra2
  38. NZBThrottle
  39. Organizr
  40. pyLoad
  41. Radarr4k
  42. Resilio
  43. rflood-openvpn
  44. rutorrent-openvpn
  45. ShareSite
  46. Shoko Anime Server
  47. SpeedTest
  48. SyncLounge
  49. Syncthing
  50. The Lounge
  51. Transmission
  52. Trakt.or
  53. Ubooquity
  54. x TeVe
  55. Zammad

Misc Info & Articles

  1. Change Server Time
  2. Fail2Ban
  3. Custom Containers
Scripts
  1. Scripting Area
    1. Auto Upload for Sonarr and Lidarr
    2. Auto Delete Log
    3. Plex Plugin DB Backup
    4. Plex fast BACKUP
Clone this wiki locally
You can’t perform that action at this time.