rclone Google Photos cannot sync the data before 2000

[ghost]

Output of rclone version

rclone v1.49.5

Describe the issue

rclone Google Photos cannot sync the data before 2000，but gphotos-sync(https://pypi.org/project/gphotos-sync/) can do this.

[ncw]

You should be able to read the media in media/all and in album.

It is quite expensive finding the oldest file in the photos album so rclone hard codes the 2000 value.

This could be a parameter? What do you think?

And the reply on the forum was:

I synced only media/all and album.
I don't know where to look for photos data before 2000（year）.

[ncw]

media/all should get you all the photos regardless of age.

This could be made configurable

rclone/backend/googlephotos/pattern.go

Line 229 in 38652d0

for year := 2000; year <= currentYear; year++ {

[ghost]

i think made it configurable maybe better… thanks

[ncw]

Do you want to try doing this? I can talk you through it.

[ghost]

Yes！Please tell me the test method.

[sakshamkhanna]

It doesn't feel right, having this in a config param.
While setting up a connection, the user shouldn't be expected to know the timestamp of the oldest photo in the library.

One way to do it could be in the googlephotos.init().
Right after registering the regInfo, make a bunch of api calls to GET media with DateFilter.Range param used like a binary search between 1800(??) and time.Now() to narrow down to the year in which the first media item appears

This seems a bit hacky, which leads to an idea for a new feature.
A callback function that gets triggered everytime a remote is added/authenticated/refreshed to enable unique workflows like this.

Any thoughts?

[ncw]

It doesn't feel right, having this in a config param.

It isn't ideal!

While setting up a connection, the user shouldn't be expected to know the timestamp of the oldest photo in the library.

One way to do it could be in the googlephotos.init().
Right after registering the regInfo, make a bunch of api calls to GET media with DateFilter.Range param used like a binary search between 1800(??) and time.Now() to narrow down to the year in which the first media item appears

Binary search is a nice idea :-)

It will take about 8 queries to do this at 1-2 second each so a noticeable amount of time.

This seems a bit hacky, which leads to an idea for a new feature.
A callback function that gets triggered everytime a remote is added/authenticated/refreshed to enable unique workflows like this.

There is one already. So this could run to update the value stored in the config file.

[ncw]

@YNZone wrote:

Yes！Please tell me the test method.

First step create a new option here, called start_year type int, default 2000, Advanced: true

rclone/backend/googlephotos/googlephotos.go

Line 137 in 22bf858

}},

Add it to the options struct

rclone/backend/googlephotos/googlephotos.go

Line 142 in 22bf858

type Options struct {

Next step use that option here

rclone/backend/googlephotos/pattern.go

Line 227 in 22bf858

func years(ctx context.Context, f lister, prefix string, match []string) (entries fs.DirEntries, err error) {

You'll need to make years a method on Fs so it can read f.opt.StartYear

[ghost]

Sorry, this is a bit difficult. I think it would be better to wait for the 1.51 version.

[sakshamkhanna]

There is one already. So this could run to update the value stored in the config file.

@ncw could point me to where the callback function is configured. I cant seem to find it

[ncw]

This is the config callback

rclone/backend/googlephotos/googlephotos.go

Line 82 in 18c029e

Config: func(name string, m configmap.Mapper) {

[darkLord19]

@ncw
Hi I just came across this project and I am interested in contributing to the project and I find this issue a good start to get in.

You'll need to make years a method on Fs so it can read f.opt.StartYear

When you say make years a method on Fs, do you mean to make years a method of interface Fs defined here?

rclone/fs/fs.go

Line 254 in ab89539

type Fs interface {

[ncw]

When you say make years a method on Fs, do you mean to make years a method of interface Fs defined here?

rclone/fs/fs.go

Line 254 in ab89539

type Fs interface {

The function years needs to be a method on the type Fs defined in the backend

rclone/backend/googlephotos/googlephotos.go

Line 148 in 1868c77

type Fs struct {

so become

 func (f *Fs) years(ctx context.Context, f lister, prefix string, match []string) (entries fs.DirEntries, err error) { 

Then instead of using the constant 2000 you can use f.opt.StartYear

rclone/backend/googlephotos/pattern.go

Line 229 in 1868c77

for year := 2000; year <= currentYear; year++ {

[ghost]

I'd like to tackle this one if no one has yet.

[ncw]

@dooven wrote:

I'd like to tackle this one if no one has yet.

That would be great - thank you :-)

[ncw]

Thanks to @dooven this is fixed!

I've merged this to master now which means it will be in the latest beta in 15-30 mins and released in v1.52