Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could not find decrypted mountpoint #35

Closed
zeyoner opened this issue Feb 10, 2018 · 165 comments
Closed

Could not find decrypted mountpoint #35

zeyoner opened this issue Feb 10, 2018 · 165 comments

Comments

@zeyoner
Copy link

zeyoner commented Feb 10, 2018

I'm sure this may be an easy fix. Also their may be a possiblity I did something wrong. However I've spent an entire day attempting to figure out the issue.

[ 2018-02-10@11:15:21 ] ###### Start Mount ###### [ 2018-02-10@11:15:26 ] Mounting Google Drive with Plexdrive in mountpoint: /opt/cloud-media-scripts/.cloud-encrypt [ 2018-02-10@11:15:26 ] Mounted successfully with Plexdrive [ 2018-02-10@11:15:31 ] Mounting decrypted Google Drive with Rclone in mountpoint: /opt/cloud-media-scripts/.cloud-decrypt [ 2018-02-10@11:15:31 ] Mounted successfully with Rclone [ 2018-02-10@11:15:36 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds. [ 2018-02-10@11:15:46 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds. [ 2018-02-10@11:15:56 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds. [ 2018-02-10@11:16:06 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds. [ 2018-02-10@11:16:16 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds. [ 2018-02-10@11:16:26 ] Could not read mountpoint: /opt/cloud-media-scripts/.cloud-decrypt [ 2018-02-10@11:16:26 ] Mount failed

What else can I provide to get this up and running?

@Dulanic
Copy link
Contributor

Dulanic commented Feb 10, 2018

Can you provide your config? Also the mount.log may help. Also the rclone.conf might be part of it... it's hard to tell off hand so far.

@zeyoner
Copy link
Author

zeyoner commented Feb 10, 2018

Here ya go.
config.txt
rclone.txt

@Dulanic
Copy link
Contributor

Dulanic commented Feb 10, 2018

So I am thinking it has more do do with your rclone.conf, I know on ls.io discord you mentioned using a Media dir. I'd recommend setting this in the config and rerunning setup, it will adjust your rclone mount points to ensure their correct.

google_drive_media_directory="" # Empty means media files are in root

This is what my rclone.conf looks like:

[gd]
type = drive
client_id = **Redacted**
client_secret = **Redacted**
token = **Redacted**

[gd-crypt]
type = crypt
remote = gd:Media
filename_encryption = standard
password =**Redacted**
password2 =
directory_name_encryption = true

[local-crypt]
type = crypt
remote = /opt/cloud-media-scripts/.cloud-encrypt/Media
filename_encryption = standard
password = **Redacted**
password2 =
directory_name_encryption = true

@Dulanic
Copy link
Contributor

Dulanic commented Feb 10, 2018

Also the setup.sh isnt built around using install_dir, but you could edit it as necessary. Also you could run the curl /w the install dir instead?

bash <( curl -Ls https://github.com/madslundt/cloud-media-scripts/raw/master/INSTALL ) [PATH]

Replace [PATH] /w /mnt/storage/cloud

@zeyoner
Copy link
Author

zeyoner commented Feb 11, 2018

Alright so I went ahead and ran the umount.remote and removed all traces of the scripts.

From /opt I ran bash <( curl -Ls https://github.com/madslundt/cloud-media-scripts/raw/master/INSTALL )
nano /opt/cloud-media-scripts/config
config.txt

bash /opt/cloud-media-scripts/setup.sh

rclone setup
rclone.txt

plexdrive setup

mount.remote
[ 2018-02-11@11:39:00 ] ###### Start Mount ######
[ 2018-02-11@11:39:05 ] Mounting Google Drive with Plexdrive in mountpoint: /opt/cloud-media-scripts/.cloud-encrypt
[ 2018-02-11@11:39:05 ] Mounted successfully with Plexdrive
[ 2018-02-11@11:39:10 ] Mounting decrypted Google Drive with Rclone in mountpoint: /opt/cloud-media-scripts/.cloud-decrypt
[ 2018-02-11@11:39:10 ] Mounted successfully with Rclone
[ 2018-02-11@11:39:15 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@11:39:25 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@11:39:35 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@11:39:45 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@11:39:55 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@11:40:05 ] Could not read mountpoint: /opt/cloud-media-scripts/.cloud-decrypt
[ 2018-02-11@11:40:05 ] Mount failed

What I have noticed is that rclone appears to have two different locations for it's rclone.conf. The one within /opt/cloud-media-scripts/rclone and theirs an other location. I noticed this because if you look at the provided rclone.conf it's from /opt/cloud-media-scripts/rclone it's missing gd

However, when I run rclone config:
`root@hoggle:/opt/cloud-media-scripts/rclone# nano rclone.conf
Use "fg" to return to nano.

[1]+ Stopped nano rclone.conf
root@hoggle:/opt/cloud-media-scripts/rclone# rclone config
Current remotes:

Name Type
==== ====
gd drive
gd-crypt crypt
local-crypt crypt

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q>
`
I've verified that the all three of the remotes are correct. My assumption is that when I first ran the Easy install and it didn't work for me. So I went ahead and attempted the manual path where I installed rclone myself. So the system now has rclone installed outside of the Easy install. I've also tried configuring the rclone.conf in /opt/cloud-media-scripts/rclone so that it's correct. Still does not mount.

Now I don't believe the setup.sh is even using the rclone.conf in /opt/cloud-media-scripts/rclone. Reason being that when I run the setup.sh and I get to the rclone setup it already has all three remotes configured even after completely removing the directory /opt/cloud-media-scripts/ and reinstalling then setup.sh the rclone configuration is still there.

@Dulanic
Copy link
Contributor

Dulanic commented Feb 11, 2018

rclong config is going to use the default rclone config location which isnt that folder. You need to do
rclone --config /opt/cloud-media-scripts/rclone/rclone.conf config
to get it to use the right one.

setup.sh does this which will replace the rclone_config from config.
${rclone_bin} --config=${rclone_config} config

rclone_config is set to

rclone_config="${rclone_dir}/rclone.conf"
rclone_dir="${install_dir}/rclone"

and your install dir was
install_dir="/opt/cloud-media-scripts"

@zeyoner
Copy link
Author

zeyoner commented Feb 11, 2018

I understand all of that. However nothing has changed. Even with me changing the location of the rclone.conf with rclone --config /opt/cloud-media-scripts/rclone/rclone.conf config it does not make a difference. As for everything else it's been set to defaults. I posted the config in the previous post.

I've run out of ideas. It refuses to find the decrypted mountpoint even though it states it's mounted successfully.

@Dulanic
Copy link
Contributor

Dulanic commented Feb 11, 2018

2 things to check....

  1. Is there anything in /opt/cloud-media-scripts/,cloud-encrypt
  2. If yes to # 1 please provide a copy of your /opt/cloud-media-scripts/rclone/rclone.conf (redact as needed for client id etc...)

@zeyoner
Copy link
Author

zeyoner commented Feb 11, 2018

It's empty.

@Dulanic
Copy link
Contributor

Dulanic commented Feb 11, 2018

Then you have deeper issues, something /w your rclone isnt right. You should see the encrypted data. You need to test your rclone further to see if its even working properly... like
rclone ls --config /opt/cloud-media-scripts/rclone/rclone.conf gd:

and see if things show. And I flipped what I said last post, can you provide

/opt/cloud-media-scripts/rclone/rclone.conf (redact as needed for client id etc...)

@zeyoner
Copy link
Author

zeyoner commented Feb 11, 2018

rclone ls --config /opt/cloud-media-scripts/rclone/rclone.conf gd: scrolled all of the junk I have on GDrive.

[gd-crypt]
type = crypt
remote = gd:
filename_encryption = standard
password =
password2 =

[local-crypt]
type = crypt
remote = /opt/cloud-media-scripts/.cloud-encrypt/
filename_encryption = standard
password =
password2 =
directory_name_encryption = true

[gd]
type = drive
client_id =
client_secret =
service_account_file =
token = Redacted

@Dulanic
Copy link
Contributor

Dulanic commented Feb 11, 2018

OK so I know you mentioned you store it all in a subdir, so you need to rerun the setup after you update your location in config.... like mine is Media. Your mount locations are wrong due to yours being "".

google_drive_media_directory="Media" # Empty means media files are in root

@zeyoner
Copy link
Author

zeyoner commented Feb 11, 2018

I've made the change to config google_drive_media_directory="Media" # Empty means media files are in root

root@hoggle:/# bash /opt/cloud-media-scripts/setup.sh
Ign:1 http://deb.debian.org/debian stretch InRelease
Get:2 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Get:3 http://deb.debian.org/debian stretch-backports InRelease [91.8 kB]
Hit:4 http://deb.debian.org/debian stretch Release
Hit:5 https://download.docker.com/linux/debian stretch InRelease
Get:6 http://deb.debian.org/debian stretch-backports/main amd64 Packages.diff/Index [27.8 kB]
Get:7 http://deb.debian.org/debian stretch-backports/main Translation-en.diff/Index [27.8 kB]
Get:8 http://deb.debian.org/debian stretch-backports/main amd64 Packages 2018-02-11-1425.25.pdiff [23.6 kB]
Get:8 http://deb.debian.org/debian stretch-backports/main amd64 Packages 2018-02-11-1425.25.pdiff [23.6 kB]
Get:9 http://deb.debian.org/debian stretch-backports/main Translation-en 2018-02-11-1425.25.pdiff [1,495 B]
Get:9 http://deb.debian.org/debian stretch-backports/main Translation-en 2018-02-11-1425.25.pdiff [1,495 B]
Get:11 http://security.debian.org/debian-security stretch/updates InRelease [63.0 kB]
Fetched 327 kB in 4s (80.8 kB/s)
Reading package lists... Done
Installing dependencies

Checking Rclone and PlexdriveRclone is already in version v1.39. Do you want to run setup anyway? [Y/n]? y
--2018-02-11 16:00:17-- https://github.com/ncw/rclone/releases/download/v1.39/rclone-v1.39-linux-amd64.zip
Resolving github.com (github.com)... 192.30.253.112, 192.30.253.113
Connecting to github.com (github.com)|192.30.253.112|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/17803236/0e0bb858-e7e6-11e7-8348-13f9448e7a80?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180211T210017Z&X-Amz-Expires=300&X-Amz-Signature=83a634c19e0f66d37e03dbc2b1c19eaa4c42b0ed35a2647e097f3d4d56a9c794&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Drclone-v1.39-linux-amd64.zip&response-content-type=application%2Foctet-stream [following]
--2018-02-11 16:00:17-- https://github-production-release-asset-2e65be.s3.amazonaws.com/17803236/0e0bb858-e7e6-11e7-8348-13f9448e7a80?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180211T210017Z&X-Amz-Expires=300&X-Amz-Signature=83a634c19e0f66d37e03dbc2b1c19eaa4c42b0ed35a2647e097f3d4d56a9c794&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Drclone-v1.39-linux-amd64.zip&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.20.195
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.20.195|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6270216 (6.0M) [application/octet-stream]
Saving to: ‘rclone-v1.39-linux-amd64.zip’

rclone-v1.39-linux-amd64.zip 100%[=======================================================>] 5.98M 5.69MB/s in 1.1s

2018-02-11 16:00:18 (5.69 MB/s) - ‘rclone-v1.39-linux-amd64.zip’ saved [6270216/6270216]

Archive: rclone-v1.39-linux-amd64.zip
inflating: /opt/cloud-media-scripts/rclone-v1.39-linux-amd64/README.html
inflating: /opt/cloud-media-scripts/rclone-v1.39-linux-amd64/rclone
inflating: /opt/cloud-media-scripts/rclone-v1.39-linux-amd64/rclone.1
inflating: /opt/cloud-media-scripts/rclone-v1.39-linux-amd64/README.txt
--2018-02-11 16:00:18-- https://github.com/dweidenfeld/plexdrive/releases/download/4.0.0/plexdrive-linux-amd64
Resolving github.com (github.com)... 192.30.253.112, 192.30.253.113
Connecting to github.com (github.com)|192.30.253.112|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/89668997/116d97dc-4d2c-11e7-851d-348db60ea8ec?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180211T210018Z&X-Amz-Expires=300&X-Amz-Signature=1cff17ca3972bf193039118bcb006218fb392a75418026418f2851625253d475&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dplexdrive-linux-amd64&response-content-type=application%2Foctet-stream [following]
--2018-02-11 16:00:18-- https://github-production-release-asset-2e65be.s3.amazonaws.com/89668997/116d97dc-4d2c-11e7-851d-348db60ea8ec?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180211T210018Z&X-Amz-Expires=300&X-Amz-Signature=1cff17ca3972bf193039118bcb006218fb392a75418026418f2851625253d475&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dplexdrive-linux-amd64&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.20.195
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.20.195|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10546476 (10M) [application/octet-stream]
Saving to: ‘plexdrive-linux-amd64’

plexdrive-linux-amd64 100%[=======================================================>] 10.06M 7.18MB/s in 1.4s

2018-02-11 16:00:20 (7.18 MB/s) - ‘plexdrive-linux-amd64’ saved [10546476/10546476]

Rclone config already exists
Do you want to change Rclone config now [Y/n]? y
=======RCLONE========

When this is done exit rclone config by pressing 'Q'
Current remotes:

Name Type
==== ====
gd drive
gd-crypt crypt
local-crypt crypt

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

Rclone has successfully been updated
Plexdrive config already exists.

Do you want to remount now [Y/n]? y
[ 2018-02-11@16:00:28 ] Unmounting decrypted cloud mountpoint -> /opt/cloud-media-scripts/.cloud-decrypt
fusermount: failed to unmount /opt/cloud-media-scripts/.cloud-decrypt: Invalid argument
[ 2018-02-11@16:00:28 ] Unmounting encrypted cloud mountpoint -> /opt/cloud-media-scripts/.cloud-encrypt
[ 2018-02-11@16:00:28 ] Unmounting final library mountpoint -> /mnt/storage/cloud/media
fusermount: failed to unmount /mnt/storage/cloud/media: Invalid argument
[ 2018-02-11@16:00:28 ] ###### Start Mount ######
[ 2018-02-11@16:00:33 ] Mounting Google Drive with Plexdrive in mountpoint: /opt/cloud-media-scripts/.cloud-encrypt
[ 2018-02-11@16:00:33 ] Mounted successfully with Plexdrive
[ 2018-02-11@16:00:38 ] Mounting decrypted Google Drive with Rclone in mountpoint: /opt/cloud-media-scripts/.cloud-decrypt
[ 2018-02-11@16:00:38 ] Mounted successfully with Rclone
[ 2018-02-11@16:00:43 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@16:00:54 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@16:01:04 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@16:01:14 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@16:01:24 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@16:01:34 ] Could not read mountpoint: /opt/cloud-media-scripts/.cloud-decrypt
[ 2018-02-11@16:01:34 ] Mount failed

@Dulanic
Copy link
Contributor

Dulanic commented Feb 11, 2018

OK so you use encrypted yet made no changes to rclone when you set it up? That looks to be a problem... you put y you want to change, but you didnt go in and set your pw or anything? You need to edit your mounts to set the passwords etc... then q.

`Do you want to change Rclone config now [Y/n]? y
=======RCLONE========
When this is done exit rclone config by pressing 'Q'
Current remotes:
Name Type
==== ====
gd drive
gd-crypt crypt
local-crypt crypt
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
`

@zeyoner
Copy link
Author

zeyoner commented Feb 11, 2018

root@hoggle:/# bash /opt/cloud-media-scripts/setup.sh
Ign:1 http://deb.debian.org/debian stretch InRelease
Hit:2 http://deb.debian.org/debian stretch-updates InRelease
Hit:3 http://deb.debian.org/debian stretch-backports InRelease
Hit:4 http://security.debian.org/debian-security stretch/updates InRelease
Hit:5 http://deb.debian.org/debian stretch Release
Hit:6 https://download.docker.com/linux/debian stretch InRelease
Reading package lists... Done
Installing dependencies

Checking Rclone and PlexdriveRclone is already in version v1.39. Do you want to run setup anyway? [Y/n]? n
Exiting setup
root@hoggle:/# cd /opt/cloud-media-scripts/scripts/
root@hoggle:/opt/cloud-media-scripts/scripts# ./mount.remote
[ 2018-02-11@16:11:28 ] ###### Start Mount ######
[ 2018-02-11@16:11:28 ] Google Drive mountpoint: /opt/cloud-media-scripts/.cloud-encrypt already mounted.
[ 2018-02-11@16:11:33 ] Mounting decrypted Google Drive with Rclone in mountpoint: /opt/cloud-media-scripts/.cloud-decrypt
[ 2018-02-11@16:11:33 ] Mounted successfully with Rclone
[ 2018-02-11@16:11:38 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@16:11:48 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@16:11:58 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@16:12:08 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@16:12:18 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@16:12:28 ] Could not read mountpoint: /opt/cloud-media-scripts/.cloud-decrypt
[ 2018-02-11@16:12:28 ] Mount failed
root@hoggle:/opt/cloud-media-scripts/scripts# rclone config
Current remotes:

Name Type
==== ====
gd drive
gd-crypt crypt
local-crypt crypt

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
root@hoggle:/opt/cloud-media-scripts/scripts#

My head is spinning at this point..

@Dulanic
Copy link
Contributor

Dulanic commented Feb 11, 2018

Not sure what you're showing? Looks like you didnt change anything. When you run setup.sh you need to go in and edit the rclone.conf and edit gd-crypt and local-crypt by settings your encryption settings.


[gd]
type = drive
client_id = *redacted*
client_secret = *redacted*
token = *redacted*

[gd-crypt]
type = crypt
remote = gd:Media
filename_encryption = standard
password = *redacted*
password2 =
directory_name_encryption = true

[local-crypt]
type = crypt
remote = /opt/cloud-media-scripts/.cloud-encrypt/Media
filename_encryption = standard
password = *redacted*
password2 =
directory_name_encryption = true


@zeyoner
Copy link
Author

zeyoner commented Feb 11, 2018

I really appreciate you helping me. But I don't understand what it is that you're asking me to do. I've redone the rclone config so many times it's ridiclious. I've done it before running setup.sh and after as well. Still same results. Maybe I'm configuring it wrong?

I followed the following:
Endpoint to your cloud storage.
Create new remote [Press N]
Give it a name (default in config gd)
Choose Google Drive [Press 8]
If you have a client id paste it here or leave it blank
Choose headless machine [Press N]
Open the url in your browser and enter the verification code
Encryption and decryption for your cloud storage.
Create new remote [Press N]
Give it the same name as the variable rclone_cloud_endpoint specified in the config but without colon (:) (default in config gd-crypt)
Choose Encrypt/Decrypt a remote [Press 5]
Enter the name of the endpoint created in cloud-storage appended with a colon (:) and the subfolder on your cloud. Example gd:/Media or just gd: if you have your files in root in the cloud.
Choose how to encrypt filenames. I prefer option 2 Encrypt the filenames
Choose to either generate your own or random password. I prefer to enter my own.
Choose to enter pass phrase for the salt or leave it blank. I prefer to enter my own.
Encryption and decryption for your local storage.
Create new remote [Press N]
Give it the same name as specified in the environment variable rclone_local_endpoint but without colon (:) (default in config local-crypt)
Choose Encrypt/Decrypt a remote [Press 5]
Enter the encrypted folder specified as cloud_encrypt_dir in config. If you are using subdirectory on Google Drive append it to it. Example /.cloud-encrypt/Media
Choose the same filename encrypted as you did with the cloud storage.
Enter the same password as you did with the cloud storage.
Enter the same pass phrase as you did with the cloud storage.

Now that step by step is outdated. So maybe I'm doing something wrong?

@Dulanic
Copy link
Contributor

Dulanic commented Feb 11, 2018

Can you post your rclone.conf now from /opt/cloud-media-scripts/rclone?

@zeyoner
Copy link
Author

zeyoner commented Feb 11, 2018

[gd-crypt]
type = crypt
remote = gd:
filename_encryption = standard
password =
password2 =

[local-crypt]
type = crypt
remote = /opt/cloud-media-scripts/.cloud-encrypt/
filename_encryption = standard
password =
password2 =
directory_name_encryption = true

[gd]
type = drive
client_id =
client_secret =
service_account_file =
token = Redacted

@Dulanic
Copy link
Contributor

Dulanic commented Feb 11, 2018

That again shows you didn't set your password during setup on the rclone setup. What I'm saying you need to do is say Y for modify rclone config, type e for edit, and set your encryption password in local-crypt and in gd-crypt. It also does not show the gd:Media etc... in the rclone.conf which it should.

During setup.sh it should add the GOOGLE_DRIVE_MEDIA_DIRECTORY after the gd: which isn't showing for you which means the config file wasnt updated to show your correct location for your encrypted dir on Gdrive.

[gd-crypt]
type = crypt
remote = gd:<GOOGLE_DRIVE_MEDIA_DIRECTORY>
filename_encryption = standard
password = <PASSWORD>
password2 = <PASSWORD2>

[local-crypt]
type = crypt
remote = <ENCRYPTED_FOLDER>/<GOOGLE_DRIVE_MEDIA_DIRECTORY>
filename_encryption = standard
password = <PASSWORD>
password2 = <PASSWORD2>

@zeyoner
Copy link
Author

zeyoner commented Feb 11, 2018

root@hoggle:/opt/cloud-media-scripts/rclone# bash /opt/cloud-media-scripts/setup.sh 
Ign:1 http://deb.debian.org/debian stretch InRelease
Get:2 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Get:3 http://deb.debian.org/debian stretch-backports InRelease [91.8 kB]               
Get:4 http://security.debian.org/debian-security stretch/updates InRelease [63.0 kB]   
Hit:5 http://deb.debian.org/debian stretch Release                                   
Hit:6 https://download.docker.com/linux/debian stretch InRelease
Get:7 http://deb.debian.org/debian stretch-updates/main Sources.diff/Index [2,704 B]
Get:8 http://deb.debian.org/debian stretch-updates/main amd64 Packages.diff/Index [2,704 B]
Get:9 http://deb.debian.org/debian stretch-updates/main Sources 2018-02-11-2045.46.pdiff [623 B]
Get:9 http://deb.debian.org/debian stretch-updates/main Sources 2018-02-11-2045.46.pdiff [623 B]
Get:10 http://deb.debian.org/debian stretch-updates/main amd64 Packages 2018-02-11-2045.46.pdiff [332 B]
Get:10 http://deb.debian.org/debian stretch-updates/main amd64 Packages 2018-02-11-2045.46.pdiff [332 B]
Fetched 252 kB in 1s (223 kB/s)                     
Reading package lists... Done
Installing dependencies


Checking Rclone and PlexdriveRclone is already in version v1.39. Do you want to run setup anyway? [Y/n]? y
--2018-02-11 17:44:52--  https://github.com/ncw/rclone/releases/download/v1.39/rclone-v1.39-linux-amd64.zip
Resolving github.com (github.com)... 192.30.253.113, 192.30.253.112
Connecting to github.com (github.com)|192.30.253.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/17803236/0e0bb858-e7e6-11e7-8348-13f9448e7a80?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180211T224452Z&X-Amz-Expires=300&X-Amz-Signature=1987fcb37fc8db9f210206f0acfacf51f90be8e2063ee557b624c7430ba2d345&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Drclone-v1.39-linux-amd64.zip&response-content-type=application%2Foctet-stream [following]
--2018-02-11 17:44:52--  https://github-production-release-asset-2e65be.s3.amazonaws.com/17803236/0e0bb858-e7e6-11e7-8348-13f9448e7a80?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180211T224452Z&X-Amz-Expires=300&X-Amz-Signature=1987fcb37fc8db9f210206f0acfacf51f90be8e2063ee557b624c7430ba2d345&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Drclone-v1.39-linux-amd64.zip&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.162.59
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.162.59|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6270216 (6.0M) [application/octet-stream]
Saving to: ‘rclone-v1.39-linux-amd64.zip’

rclone-v1.39-linux-amd64.zip          100%[======================================================================>]   5.98M  6.01MB/s    in 1.0s    

2018-02-11 17:44:53 (6.01 MB/s) - ‘rclone-v1.39-linux-amd64.zip’ saved [6270216/6270216]

Archive:  rclone-v1.39-linux-amd64.zip
  inflating: /opt/cloud-media-scripts/rclone-v1.39-linux-amd64/README.html  
  inflating: /opt/cloud-media-scripts/rclone-v1.39-linux-amd64/rclone  
  inflating: /opt/cloud-media-scripts/rclone-v1.39-linux-amd64/rclone.1  
  inflating: /opt/cloud-media-scripts/rclone-v1.39-linux-amd64/README.txt  
--2018-02-11 17:44:54--  https://github.com/dweidenfeld/plexdrive/releases/download/4.0.0/plexdrive-linux-amd64
Resolving github.com (github.com)... 192.30.253.113, 192.30.253.112
Connecting to github.com (github.com)|192.30.253.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/89668997/116d97dc-4d2c-11e7-851d-348db60ea8ec?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180211T224454Z&X-Amz-Expires=300&X-Amz-Signature=92bc3bf5b0aa9d6b0c4eb4f578845c55c4db6f1de042eb5775e7cd69ceeb6d98&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dplexdrive-linux-amd64&response-content-type=application%2Foctet-stream [following]
--2018-02-11 17:44:54--  https://github-production-release-asset-2e65be.s3.amazonaws.com/89668997/116d97dc-4d2c-11e7-851d-348db60ea8ec?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180211T224454Z&X-Amz-Expires=300&X-Amz-Signature=92bc3bf5b0aa9d6b0c4eb4f578845c55c4db6f1de042eb5775e7cd69ceeb6d98&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dplexdrive-linux-amd64&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.162.59
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.162.59|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10546476 (10M) [application/octet-stream]
Saving to: ‘plexdrive-linux-amd64’

plexdrive-linux-amd64                 100%[======================================================================>]  10.06M  7.54MB/s    in 1.3s    

2018-02-11 17:44:56 (7.54 MB/s) - ‘plexdrive-linux-amd64’ saved [10546476/10546476]

Rclone config already exists
Do you want to change Rclone config now [Y/n]? y
=======RCLONE========


When this is done exit rclone config by pressing 'Q'
Current remotes:

Name                 Type
====                 ====
gd                   drive
gd-crypt             crypt
local-crypt          crypt

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> e
Choose a number from below, or type in an existing value
 1 > gd
 2 > gd-crypt
 3 > local-crypt
remote> 2
--------------------
[gd-crypt]
type = crypt
remote = gd:
filename_encryption = standard
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***
--------------------
Edit remote
Value "remote" = "gd:"
Edit? (y/n)>
y) Yes
n) No
y/n> n
Value "filename_encryption" = "standard"
Edit? (y/n)>
y) Yes
n) No
y/n> n
Value "directory_name_encryption" = ""
Edit? (y/n)>
y) Yes
n) No
y/n> y
Option to either encrypt directory names or leave them intact.
Choose a number from below, or type in your own value
 1 / Encrypt directory names.
   \ "true"
 2 / Don't encrypt directory names, leave them intact.
   \ "false"
directory_name_encryption> 1
Value "password" = "<PASSWORD>"
Edit? (y/n)>
y) Yes
n) No
y/n> y
Password or pass phrase for encryption.
y) Yes type in my own password
g) Generate random password
y/g> y
Enter the password:
password:
Confirm the password:
password:
Value "password2" = "<PASSWORD2>"
Edit? (y/n)>
y) Yes
n) No
y/n> y
Password or pass phrase for salt. Optional but recommended.
Should be different to the previous password.
y) Yes type in my own password
g) Generate random password
n) No leave this optional password blank
y/g/n> @LI@joke!
y/g/n> y
Enter the password:
password:
Confirm the password:
password:
--------------------
[gd-crypt]
type = crypt
remote = gd:
filename_encryption = standard
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***
directory_name_encryption = true
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
Remote config
Current remotes:

Name                 Type
====                 ====
gd                   drive
gd-crypt             crypt
local-crypt          crypt

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> e
Choose a number from below, or type in an existing value
 1 > gd
 2 > gd-crypt
 3 > local-crypt
remote> 3
--------------------
[local-crypt]
type = crypt
remote = /opt/cloud-media-scripts/.cloud-encrypt/
filename_encryption = standard
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***
directory_name_encryption = true
--------------------
Edit remote
Value "remote" = "/opt/cloud-media-scripts/.cloud-encrypt/"
Edit? (y/n)>
y) Yes
n) No
y/n> n
Value "filename_encryption" = "standard"
Edit? (y/n)>
y) Yes
n) No
y/n> n
Value "directory_name_encryption" = "true"
Edit? (y/n)>
y) Yes
n) No
y/n> n
Value "password" = "<PASSWORD>"
Edit? (y/n)>
y) Yes
n) No
y/n> y
Password or pass phrase for encryption.
y) Yes type in my own password
g) Generate random password
y/g> y
Enter the password:
password:
Confirm the password:
password:
Value "password2" = "<PASSWORD2>"
Edit? (y/n)>
y) Yes
n) No
y/n> y
Password or pass phrase for salt. Optional but recommended.
Should be different to the previous password.
y) Yes type in my own password
g) Generate random password
n) No leave this optional password blank
y/g/n> y
Enter the password:
password:
Confirm the password:
password:
--------------------
[local-crypt]
type = crypt
remote = /opt/cloud-media-scripts/.cloud-encrypt/
filename_encryption = standard
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***
directory_name_encryption = true
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
Remote config
Current remotes:

Name                 Type
====                 ====
gd                   drive
gd-crypt             crypt
local-crypt          crypt

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

Rclone has successfully been updated
Plexdrive config already exists.


Do you want to remount now [Y/n]? y
[ 2018-02-11@17:47:07 ] Unmounting decrypted cloud mountpoint -> /opt/cloud-media-scripts/.cloud-decrypt
fusermount: failed to unmount /opt/cloud-media-scripts/.cloud-decrypt: Invalid argument
[ 2018-02-11@17:47:07 ] Unmounting encrypted cloud mountpoint -> /opt/cloud-media-scripts/.cloud-encrypt
[ 2018-02-11@17:47:07 ] Unmounting final library mountpoint -> /mnt/storage/cloud/media
fusermount: failed to unmount /mnt/storage/cloud/media: Invalid argument
[ 2018-02-11@17:47:07 ] ###### Start Mount ######
[ 2018-02-11@17:47:12 ] Mounting Google Drive with Plexdrive in mountpoint: /opt/cloud-media-scripts/.cloud-encrypt
[ 2018-02-11@17:47:12 ] Mounted successfully with Plexdrive
[ 2018-02-11@17:47:17 ] Mounting decrypted Google Drive with Rclone in mountpoint: /opt/cloud-media-scripts/.cloud-decrypt
[ 2018-02-11@17:47:17 ] Mounted successfully with Rclone
[ 2018-02-11@17:47:22 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@17:47:32 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@17:47:42 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@17:47:52 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@17:48:02 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-11@17:48:12 ] Could not read mountpoint: /opt/cloud-media-scripts/.cloud-decrypt
[ 2018-02-11@17:48:12 ] Mount failed

@Dulanic
Copy link
Contributor

Dulanic commented Feb 11, 2018

Can you give me your config now? Something is still not looking right.

@zeyoner
Copy link
Author

zeyoner commented Feb 11, 2018

#!/bin/bash
# shellcheck disable=SC2034
###############################################################################
# DIRECTORIES
###############################################################################
media_dir="/opt/cloud-media-scripts"

# Cloud directories
cloud_encrypt_dir="${media_dir}/.cloud-encrypt"
cloud_decrypt_dir="${media_dir}/.cloud-decrypt"

# Local directory
local_decrypt_dir="${media_dir}/.local-decrypt/Media"
plexdrive_temp_dir="${media_dir}/.local-decrypt/plexdrive"

# Media directory (FINAL)
local_media_dir="/mnt/storage/cloud/media"


# Encrypt media (0 means no encryption)
encrypt_media=1


###############################################################################
# UNION-FS
###############################################################################
ufs_bin="/usr/bin/unionfs"

# Mount user Id's. Replace $(id with a different ID if you need to override.
uid=$(id -u)
gid=$(id -g)

ufs_options="-o cow,allow_other,direct_io,nonempty,auto_cache,sync_read,uid=$uid,gid=$gid"


###############################################################################
# PLEXDRIVE
###############################################################################
plexdrive_dir="${media_dir}/plexdrive"
plexdrive_bin="${plexdrive_dir}/plexdrive-linux-amd64"

mongo_database="plexdrive"
mongo_host="localhost"
mongo_user=""
mongo_password=""

plexdrive_options="--temp=${plexdrive_temp_dir} -o allow_other,read_only --clear-chunk-max-size=300G --clear-chunk-age=24h --chunk-size=10M"


###############################################################################
# RCLONE
###############################################################################
rclone_dir="${media_dir}/rclone"
rclone_bin="${rclone_dir}/rclone"

rclone_config="${rclone_dir}/rclone.conf"
rclone_options="--buffer-size 500M --checkers 16"
rclone_mount_options="${rclone_options} --read-only --allow-non-empty --allow-other --max-read-ahead 30G"

# Rclone endpoints
rclone_cloud_endpoint="gd-crypt:"
rclone_local_endpoint="local-crypt:"

google_drive_media_directory="Media" # Empty means media files are in root


###############################################################################
# Plex (empty trash)
###############################################################################
plexdrive_dir="${media_dir}/plexdrive"
plexdrive_bin="${plexdrive_dir}/plexdrive-linux-amd64"

mongo_database="plexdrive"
mongo_host="localhost"
mongo_user=""
mongo_password=""

plexdrive_options="--temp=${plexdrive_temp_dir} -o allow_other,read_only --clear-chunk-max-size=300G --clear-chunk-age=24h --chunk-size=10M"


###############################################################################
# RCLONE
###############################################################################
rclone_dir="${media_dir}/rclone"
rclone_bin="${rclone_dir}/rclone"

rclone_config="${rclone_dir}/rclone.conf"
rclone_options="--buffer-size 500M --checkers 16"
rclone_mount_options="${rclone_options} --read-only --allow-non-empty --allow-other --max-read-ahead 30G"

# Rclone endpoints
rclone_cloud_endpoint="gd-crypt:"
rclone_local_endpoint="local-crypt:"

google_drive_media_directory="Media" # Empty means media files are in root


###############################################################################
# Plex (empty trash)
###############################################################################
plex_url="http://localhost:32400"
plex_token="dmGLYzzQXzg3hUU5nycK"


###############################################################################
# MISC. CONFIG
###############################################################################
date_format="+%F@%T"

# Select to remove media files based on 'time', 'space' or 'instant'
#remove_files_based_on="instant"
#remove_files_based_on="time"
remove_files_based_on="space"


# Remove media files based on 'time'
remove_files_older_than=60 # In days

# Remove media files based on 'space'
remove_files_when_space_exceeds=500 # In Gigabytes
freeup_atleast=300  # In Gigabytes

@Dulanic
Copy link
Contributor

Dulanic commented Feb 11, 2018

For some reason the .sh script it not pulling in your Media folder.

Does your setup.sh have the config in the proper place?


########## CONFIGURATION ##########
. "/opt/cloud-media-scripts/config"
###################################

@zeyoner
Copy link
Author

zeyoner commented Feb 11, 2018


#!/bin/bash

########## CONFIGURATION ##########
. "/opt/cloud-media-scripts/config"
###################################
########## DOWNLOADS ##########
# Rclone
_rclone_version="v1.39"
_rclone_release="rclone-${_rclone_version}-linux-amd64"
_rclone_zip="${_rclone_release}.zip"
_rclone_url="https://github.com/ncw/rclone/releases/download/${_rclone_version}/${_rclone_zip}"

# Plexdrive
_plexdrive_bin="plexdrive-linux-amd64"
_plexdrive_url="https://github.com/dweidenfeld/plexdrive/releases/download/4.0.0/${_plexdrive_bin}"
###################################

@Dulanic
Copy link
Contributor

Dulanic commented Feb 11, 2018

Something is going wrong and I'm not sure what. Have you tried the manual process? However, the only things I see wrong are the rclone mounts. Try to edit the rclone.conf and the folders as the template indicates:

[gd-crypt]
type = crypt
remote = gd:Media
filename_encryption = standard
password =
password2 =

[local-crypt]
type = crypt
remote = /opt/cloud-media-scripts/.cloud-encrypt/Media
filename_encryption = standard
password =
password2 =

@zeyoner
Copy link
Author

zeyoner commented Feb 12, 2018

Made the changes to rclone.conf and ran ./mount.remote

Same issue.

@Dulanic
Copy link
Contributor

Dulanic commented Feb 12, 2018

Honestly, cant tell you whats going on. Doesn't make sense to me. Maybe @madslundt has an idea because Im out of them.

@zeyoner
Copy link
Author

zeyoner commented Feb 12, 2018

Yea same here. I've been at it for almost two days now. I don't have a clue as to what preventing it from finding decrypted mountpoint. I'd really like to get this working so I can eliminate my eight hard drives or at least have less hard drives. Plus not having to worry so much about backups.

@Dulanic
Copy link
Contributor

Dulanic commented Feb 12, 2018

Try running this and see if it errors? You may need to do a 2nd ssh to test. Will at least test if its a rclone issue,

rclone mount --config /opt/cloud-media-scripts/rclone/rclone.conf --buffer-size 500M --checkers 16 --read-only --allow-non-empty --allow-other --max-read-ahead 30G gd-crypt: /opt/cloud-media-scripts/.cloud-decrypt

@zeyoner
Copy link
Author

zeyoner commented Feb 22, 2018

Also I've got over 8TB of content that will need to go up. Any recommendations? any concerns?

The cron jobs are in the cron file, you just need to add them.

I knew that! lol just copy and paste everything in the cron file?

@Dulanic
Copy link
Contributor

Dulanic commented Feb 22, 2018

OK so if you have 8TB, GDrive has a 750GB a day upload cap. What I did was limit it to 8MB/sec and ran it manually the 1st time. After that, you can use the cron normally.

Just add --bwlimit=8M I think will ,limit you to around 700GB a day or so....

@zeyoner
Copy link
Author

zeyoner commented Feb 22, 2018

Not too concerned with limiting I can only upload at 35 Mbps.

@zeyoner
Copy link
Author

zeyoner commented Feb 22, 2018

So the idea would be place whatever content I want in the cloud into /mnt/storage/cloud/media and at some point it will upload. Should I also be using /mnt/storage/cloud/media to access the content? Will it display all of the content that gets upped? Would I map to it?

@Dulanic
Copy link
Contributor

Dulanic commented Feb 22, 2018 via email

@zeyoner
Copy link
Author

zeyoner commented Feb 22, 2018

I see so it would be rclone copy --config /opt/cloud-media-scripts/rclone/rclone.conf LOCATION OF CONTENT YOU WANT UP gd-crypt:

Another question:

# Mount user Id's. Replace $(id with a different ID if you need to override.
uid=$(id -u)
gid=$(id -g)

What part am I to replace? Should it be like this:
uid=1000
gid=1000

I was also wondering how does the script decide what's cached?

@Dulanic
Copy link
Contributor

Dulanic commented Feb 22, 2018

If it's working don't replace anything. I had to use my user ID because I was having permission issues. In my case I used 1000. The default is uid=$(id -u) which means it's the user running the script which may work fine for you, it seems to work for most.

So yes, it would be uid=1234 etc....

Your command should work fine for copying also.

Cached is a different topic. Cache is based on what your cofig sets. Cache just pulls down data based on what's being accessed and keeps it for a specific amount of time and/or size. Cache is primarily used for overall file structure so that, for example, a plex library scan won't get your a API ban. Another example is if someone starts watching movie A, it would cache some/all of that movie.

Basic idea is cache is mostly reactive not proactive if that makes sense. No way for plexdrive to say hey I think they will play this movie today.

@zeyoner
Copy link
Author

zeyoner commented Feb 22, 2018

Understood, I like the idea of size over time. I think at least. As for the user ID I asked because I was having permission issues with creating a folder in the local media directory. I made the change and ran the setup again the issue does not exist anymore. Plan isn't really to use Plex actually. I am more of a Kodi user not big on sharing my media nor watching it remotely. Once and awhile I stream family videos but mostly local use.

@Dulanic
Copy link
Contributor

Dulanic commented Feb 22, 2018

FYI, the uid/gid in config can be changed at any time and all you need to do is remount. It doesnt require a full setup again.

@zeyoner
Copy link
Author

zeyoner commented Feb 25, 2018

Hey guys, I'm back..

So I've been learning my way around the scripts. I noticed something today. I use both Windows and Linux in my environment. When I access the local_media_directory from Windows everything appears and I am able to access it with no issues. However I have the same share setup on my Linux workstation and I am not able to see anything. As if it's not mounted. The one different between the two is that with Windows I am using Samba and with Linux I am using NFS.

I have it set to automount with an entire in fstab.

Also had quick question. I plan in the near future to get rid of a lot of the hard drives I have once the data has been upped. I'll then be building a new and smaller server. When I set everything back up again. As long as I use the same password for rclone I should still be able to gain access to the encrypted data, correct?

@Dulanic
Copy link
Contributor

Dulanic commented Feb 26, 2018

Yes, password is all that is needed. I moved all of my data off my hard drives myself and started using my hard drives to rent out /w Storj to make a couple bucks. Might want to consider that :) I didnt want to get rid of them based on the basic idea that eventually google may stop this.

Also, you may need to change your samba share to make sure you use the same ID so it has permission to read the files.

[media]
	path = /local-media/Media
	public = yes
	force user = user
	guest account = user
	writeable = yes
	force group = group * I use root, but may need to use something else

@zeyoner
Copy link
Author

zeyoner commented Feb 26, 2018

Even though the issue isn't with Samba?

Storj huh. I'll have to look into that.

@Dulanic
Copy link
Contributor

Dulanic commented Feb 26, 2018

Maybe I mixed it up, still might want to make sure the linux share is properly setup /w the right users. I use samba on linux not NFS so cant help there, but it sounds like a permission issue.

@zeyoner
Copy link
Author

zeyoner commented Mar 7, 2018

@Dulanic ,
I've been transferring a lot of content up to the cloud. My containers are unable to view the content even though I know it's mounted because I am able to view it from my workstation. All containers are using the same UID as the script (1000) I use docker compose and all containers have the following:

    environment:
      - PUID=1000
      - PGID=1000

When I mount /mnt/storage/cloud/media as the user 1000 I have no issues. It seems to be with the containers. Any ideas?

@Dulanic
Copy link
Contributor

Dulanic commented Mar 7, 2018 via email

@zeyoner
Copy link
Author

zeyoner commented Mar 7, 2018

I see, I guess I'll try it outside of a container.

@zeyoner
Copy link
Author

zeyoner commented Mar 8, 2018

@reboot sleep 20 && /opt/cloud-media-scripts/scripts/mount.remote > /opt/cloud-media-/opt/cloud-media-scripts/scripts/logs/mount.log
@reboot sleep 40 && /opt/cloud-media-scripts/scripts/mountcheck > /opt/cloud-media-/opt/cloud-media-scripts/scripts/logs/mountcheck.log
30 3 * * 1,3,4,5,6,7 /opt/cloud-media-scripts/scripts/cloudupload > /opt/cloud-media-/opt/cloud-media-scripts/scripts/logs/cloudupload.log
30 3 * * 2 /opt/cloud-media-scripts/scripts/cloudupload > /opt/cloud-media-/opt/cloud-media-scripts/scripts/logs/cloudupload.log && /opt/cloud-me$
@hourly /opt/cloud-media-scripts/scripts/mountcheck > /opt/cloud-media-/opt/cloud-media-scripts/scripts/logs/mountcheck.log
0 */12 * * * /opt/cloud-media-scripts/scripts/empty.trash > /opt/cloud-media-/opt/cloud-media-scripts/scripts/logs/emptytrash.log

So I've pasted this into crontab. However it doesn't seems to be doing it's job. Am I suppose to add/change anything?

@Dulanic
Copy link
Contributor

Dulanic commented Mar 8, 2018

If you aren't comfortable setting cron jobs, try installing webmin, it gives you a GUI for setting cron jobs.

This is my cron setup, but I use a custom script (cloudmove) and I check the mount points every 5 minutes.

@reboot /opt/cloud-media-scripts/scripts/mount.remote >> /opt/cloud-media-scripts/logs/mount.log 2>&1
@daily /opt/cloud-media-scripts/scripts/cloudmove >> /opt/cloud-media-scripts/logs/cloudmove.log 2>&1
@weekly /opt/cloud-media-scripts/scripts/rmlocal >> /opt/cloud-media-scripts/logs/rmlocal.log 2>&1
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /opt/cloud-media-scripts/scripts/mountcheck >> /opt/cloud-media-scripts/logs/status.log 2>&1

@daily /opt/cloud-media-scripts/scripts/cloudupload >> /opt/cloud-media-scripts/logs/cloudupload.log 2>&1
4 16 * * * /etc/webmin/package-updates/update.pl
2 1,12,19 * * * /opt/scripts/restartplex >> /var/log/restartplex.log #Restart plex when not playing

@zeyoner
Copy link
Author

zeyoner commented Mar 12, 2018

Is there an easy way of verifying what has already been uploaded? I tried rclone ls --config /opt/cloud-media-scripts/rclone/rclone.conf gd: but that shows the encrypted data and its being scrolled so even if it wasn't encrypt it's going it would be difficult to view.

Also if I run the rmlocal script. Will that remove the local content which has already been uploaded?

@Dulanic
Copy link
Contributor

Dulanic commented Mar 13, 2018

You can look into .cloud-decrypt if you want to see whats been uploaded for sure.

@zeyoner
Copy link
Author

zeyoner commented Mar 14, 2018

I don't like assuming. Does the rmlocal script remove content that has already been uploaded?

@Dulanic
Copy link
Contributor

Dulanic commented Mar 14, 2018 via email

@Dulanic
Copy link
Contributor

Dulanic commented Mar 16, 2018

FYI, the cloudupload script may not be working as it was until PR is migrated. git update may not work, so you might want to remove the extra spaces manually.

#53

@zeyoner
Copy link
Author

zeyoner commented Mar 16, 2018

I was noticing an issue where it would upload then not upload. Which was why I had asked how can I view what has already been upload so that I can keep some sort of track. Is there something specific I should do?

@Dulanic
Copy link
Contributor

Dulanic commented Mar 16, 2018

Yeah we don't log by file. I plan to PR that option. Check my above linked PR and try manually deleting those extra spaces.

@Dulanic
Copy link
Contributor

Dulanic commented Mar 16, 2018

To update the scripts since the PR went in, bash INSTALL from the main dir.

@zeyoner
Copy link
Author

zeyoner commented Mar 16, 2018 via email

@Dulanic
Copy link
Contributor

Dulanic commented Mar 16, 2018 via email

@zeyoner
Copy link
Author

zeyoner commented Mar 18, 2018

I rebuild my server with a different systemboard, cpu, etc. After installing Debian and configuring everything else. I'd like to report the updated script worked out of the box. Great JOB guys! 👍 There was no need to perform any of the changes I had to do before. Everything appears to be working :) 💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants