Skip to content

GCE Feeder Box Tutorial

Admin9705 edited this page Feb 22, 2019 · 94 revisions

CLICKING the STAR in the UPPER RIGHT promotes our PROJECT on GITHUB! - WARNING, PLEASE VISIT https://pgblitz.com/wikis/ - This is the old wiki!


NOTE: YOU MUST BE USING PGDEV 5.080+ for now

Table of Contents: GCEFB


About GCEFB

  • Special Thanks : John Doe for his work and ingenuity and Beta Testing SuperTransfer2.
  • CleanUp Author : Admin9705 - Yes, I was blue in the face scripting more and cleaning :D
  • Original Poster: xxx
  • Community : Thank you for BETA Testing & Schooling Us to get this right for everyone!


Compute Engine

Basically fill out here what it is


Establishing Billing & Basic Parameters

  1. Ensure to start off by selecting the right category.

  1. When the page first loads, start off by selecting the Compute Engine.

  1. If you have not enabled your billing, you will be prompted to do so. Be careful with your usage and understand that you can be liable for any money owed.

  1. Create a Compute Engine instance:

  1. Recommend Specifications are as Followed!
  • Zone: (pick one closest to your usenet server):
    • us-central1-a (Iowa)
    • europe-west1-b (Belgium)
    • asia-east1-a (Taiwan)
    • we are picking these locations because they have 2.6ghz cores instead of 2.2ghz
  • 8-12 Cores
    • If you go above 8 cores, you will have to upgrade your account
    • This means you will have to remember to cancel your server when credits run out
  • 7-11G Memory
  • Ubuntu 16.04 LTS
    • Boot disk: 10G
    • If google added NVMe drivers to 18.04 by the time you are reading this, choose 18.04 instead.
  1. Select Local SSD Scratch Disk. To start, the option is somewhat hidden; but luckily we have a picture here for you where to start.

  1. From here, you going to create x2 Local SSD Scratch Disks. Make sure that you CREATE 2 and make sure they are NVMe versions!

Optional Security

Enter in your public ssh key so you can access the box through your own ssh client.

When you are finished with your establishing your Compute Engine, visit the front page and you should see the something of the following!


Installing PlexGuide GCE Feeder Edition

  1. First install PlexGuide by visiting here: Install PlexGuide

  2. Go through the Install Process and Select the GCE Feeder Edition. If no errors, the program will deploy most of your programs by default!

WARNING - BONEHEAD ALERT - WARNING:

If you get this error below afterwards, it means there is a 99.9 percent chance that you FAILED to create the 2 additional drives that are required.


Configuring Your Static IP & Firewall

First, you must configure your network. Please view the picture below on what to click.

Next, you have to setup your external IP address. If you fail to do this, you will not be able to access the programs!

From there, you will click a top to reserve a static ip address.

Next, you will click firewall rules and add a new one.

Ensure to setup you information as shown below. In regards to the ports for better security, you can copy this line: tcp:9000; tcp:7878; tcp:8989; tcp:6789; tcp:8080; tcp:8000; tcp:7999

If you are having problems, just select the all option; but just keep in mind that this results in poor security.

Now test by going to your programs. IPv4:9000 should work by default due to it being Portainer. If you just applied the rules, wait up-to 1 minute for the modified Google Firewall rules to kick in.


Configuring Your Compute Engine

Make a Teamdrive at drive.google.com and name it media. [Instructions]

Note: If you cannot create a teamdrive, do not see it, or have errors; [Follow This]

Configure Initial Team Drive

In it, create your directories: movies and tv Configure rclone with the teamdrive: rclone config

  1. n/s/q> n

  2. name> teamdrive

  3. storage> type --> drive

  1. client_id> [PRESS ENTER]

  2. client_secret> [PRESS ENTER]

  3. scope> 1

  4. root_folder_id> [PRESS ENTER]

  5. service_account_file> [PRESS ENTER]

  6. headless browser question y/n > n

  7. enter verification code> [PASTE FROM BROWSER]

  8. configure as a Team Drive y/n > y

  9. enter a Team Drive ID> Usually 1, but check to make sure it lines up with media

  1. y to the config if all looks right to you!

  2. q to quit and exit from the rclone config.

Verify That It Works!

Verify that your teamdrive is configured properly with rclone touch teamdrive:/test123 && echo "rclone OK" You should see a file called test123 in your Media teamdrive at drive.google.com/

RClone Cache

Configure rclone cache with your teamdrive with rclone config

name> rcache
Storage> cache
remote> teamdrive:
[ skip plex related questions ]
chunk_size> 10M
info_age> 1h
chunk_total_size> 500M

Install rclone cache & unionfs service daemons: ansible-playbook /opt/plexguide/ansible/plexguide.yml --tags rcache

Verify rclone cache is mounted with systemctl status rcache. If it isn't, check the error message and restart it with:

systemctl enable rcache systemctl restart rcache

Verify that there are teamdrive directories you made eariler in /mnt/rcache

Verify unionfs is mounted with systemctl status unionfsc. If it isn't, check the error message and restart it with systemctl restart unionfsc.


Utilizing SuperTransfer2

Before you start this process, ensure that you utilize CloudCMD. By default, this program will deploy and will be installed for you! CloudCMD allows you to upload your files to your server just by dragging and dropping your files! Visit http://your-ipv4:7999. The default username and password is plex | guide

NOTE: You can change the username and password if you wish once your inside the program. For more information, visit the [CloudCMD Wiki Page]

Load PlexGuide and Goto PG SuperTransfer2. Now press any key and enter y to start the configuration process.

Load up http://yourip:8000 and you will see where you will upload your json files. Follow the information below to create your JSON Files

Create JSON Files & Upload using CLOUDCMD

Visit: https://github.com/PGBlitz/PGBlitz.com/wiki/ST2-JSON-Files-&-CloudCMD-Upload

Follow the instructions properly!


Conclusion

If you followed everything.... (fill later)


Troubleshooting

Troubleshooting:

  1. Make sure user 1000 can read and write into /root/.config/rclone/rclone.conf
  2. Make sure there is a proper config in /root/.config/rclone/rclone.conf (sometimes, the config ends up in ~/.config/rclone/rclone.conf)
  3. Make sure the line user_allow_other is uncommented in /etc/fuse.conf


PG Forum Support: GCEFB

https://pgblitz.com/forums/pg-3rd-party-programs.148/


Visit our Website By Click the Picture Below! N|Demo

N|Solid N|Solid


Supertransfer2

Pre-requisite

**Google Drive Service Accounts ** Go to http://console.developers.google.com and navigate to the credentials area to create new credentials - the quickest way is through the 'Manage Service Accounts' page.

Service Account Name: name account (e.g. GSA1, GSA2 etc) Role: leave blank Check 'Furnish New Private Key' Check 'JSON' under key type Check 'Enable G Suite Domain-wide Delegation'

And save - this will download the JSON to your HDD. Repeat this 10-50 times (that's plenty really)

Configure SuperTransfer2

  • Optional - Install BBR Kernel Mod from the Plexguide menu under Settings and Tools and select Kernel Mods then its Option 1.

---- OLD NOTES

SSH into your box and switch to root with sudo su

Update your server with apt-get update && apt-get upgrade -y

Mount your first NVMe drive to /mnt, and the other to /nvme2. We will be using /mnt as the directory to download files to, and /nvme2 as the directory that files are unpacked to and uploaded to gdrive. (note: we will be symlinking directories in /nvme2, not writing to it directly)

Make sure you have both NVMe disks on your server with lsblk | grep nvme. You should see two unmounted NVMEe disks named nvme0n1 and nvme0n2.

Mount your NVMe disks.

mkfs.ext4 -F /dev/nvme0n1
mount -o discard,defaults,nobarrier /dev/nvme0n1 /mnt
chmod a+w /mnt
echo UUID=`blkid | grep nvme0n1 | cut -f2 -d'"'` /mnt ext4 discard,defaults,nobarrier,nofail 0 2 | tee -a /etc/fstab

mkdir -p /nvme2
mkfs.ext4 -F /dev/nvme0n2
mount -o discard,defaults,nobarrier /dev/nvme0n2 /nvme2
chmod a+w /nvme2
echo UUID=`blkid | grep nvme0n2 | cut -f2 -d'"'` /nvme2 ext4 discard,defaults,nobarrier,nofail 0 2 | tee -a /etc/fstab

Verify if they mounted properly.

ls /mnt | grep -q 'lost+found' && echo "nvme1 mount OK!" || echo "nvme1 mount FAIL"
ls /nvme2 | grep -q 'lost+found' && echo "nvme2 mount OK!" || echo "nvme2 mount FAIL"

Install Plexguide with wget -qO- pgblitz.com/install.sh | bash

- Settings:
  - No second HDD

- Install:
  - Sonarr
  - Radarr
  - Sabnzbd

Verify the ports are open: for port in 7878 8989 8080 9000; do nc -w1 -zv $(curl -s icanhazip.com) $port; done

Create passwords for each app and restart the containers: for container in portainer radarr sonarr sabnzbd; do docker restart $container; done

Make a symlink for /mnt/move and /mnt/sab/complete to /nvme/... This will allow usenet articles to unpack and upload a little faster, since it's on a separate disk. You can repeat this process with nzbget, if you prefer it over sabnzbd.

mv /mnt/move /nvme2/move
ln -s /nvme2/move /mnt

mkdir -p /nvme2/sab/complete
rm -r /mnt/sab/complete
ln -s /nvme2/sab/complete /mnt/sab/

chown -R 1000:1000 /mnt
chown -R 1000:1000 /nvme2

NOTE: no menu items for these yet. Please install manually.

Install rclone beta: curl -s https://rclone.org/install.sh | bash -s beta

chown -R 1000:1000 /root chmod 775 /root/.config/rclone/rclone.conf

http://PGBlitz.wiki

Useful Starter Links

  1. Introduction
  2. Blitz YouTube Channel
  3. G-Suite Signup
  4. Recommended NewsHosts

Prior Planning 101

  1. Recommended Pre-Reading
  2. Server - Storage Planning
  3. Usenet or BitTorrent
  4. PG Editions
  5. PGBlitz Repos
  6. Common Issues

Deploy & Config

PGBlitz
  1. SSH Server Access
  2. Create a SUDO User
  3. Install PGBlitz
  4. WatchTower
  5. Remote Path Mappings

Data Transport
  1. PG Clone
    1. Google OAuth Keys
    2. PG Move
    3. PG Blitz
    4. 2nd HD Option

Key Components

  1. Traefik
  2. Port Guard
  3. PG Shield
  4. PG Press
  5. Google Cloud - GCE
    1. Automated setup
  6. Hetzner Cloud
  7. PG Fork
  8. Extra PG Commands

Blitz App Info

  1. Accessing PG Apps
  2. App Port Scheme
  3. Plex to AutoScan

Tools & Services

Core Apps [Expand]
  1. Emby
  2. Jackett
  3. Jellyfin
  4. LazyLibrarian
  5. Lidarr
  6. Netdata
  7. NZBGet
  8. Ombi
  9. Plex
  10. Portainer
  11. qBittorrent
  12. Radarr
  13. RuTorrent
  14. SABNZBD
  15. Sonarr
  16. Tautulli
Community Apps [Expand]
  1. AirSonic
  2. AllTube
  3. Bazarr
  4. Beets
  5. Bitwarden
  6. BookSonic
  7. cAdvisor
  8. Cloud Commander
  9. DDClient
  10. Deezloaderremix
  11. Deluge
  12. DelugeVPN
  13. Duplicati
  14. EmbyStats
  15. FlexGet
  16. Gazee
  17. Gitea
  18. Handbrake
  19. Headphones
  20. Heimdall
  21. HomeAssistant
  22. HTPCManager
  23. JDownloader2
  24. jd2-openvpn
  25. Kitana
  26. Logarr
  27. MakeMKV
  28. MariaDB
  29. McMyadmin
  30. MEDUSA
  31. Mellow
  32. Monitorr
  33. Muximux
  34. Mylar
  35. NextCloud
  36. NowShowing
  37. NZBHydra2
  38. NZBThrottle
  39. Organizr
  40. pyLoad
  41. Radarr4k
  42. Resilio
  43. rflood-openvpn
  44. rutorrent-openvpn
  45. ShareSite
  46. Shoko Anime Server
  47. SpeedTest
  48. SyncLounge
  49. Syncthing
  50. The Lounge
  51. Transmission
  52. Trakt.or
  53. Ubooquity
  54. x TeVe
  55. Zammad

Misc Info & Articles

  1. Change Server Time
  2. Fail2Ban
  3. Custom Containers
Scripts
  1. Scripting Area
    1. Auto Upload for Sonarr and Lidarr
    2. Auto Delete Log
    3. Plex Plugin DB Backup
    4. Plex fast BACKUP
Clone this wiki locally
You can’t perform that action at this time.