[feature-request] command for (re-)creating missing indices for files present both locally and remotely

[js-ojus]

This is in relation to #236.

It will be quite useful to have a command that checks for missing indices, and (re-)creates them as appropriate. The following table examines some possible scenarios. In all cases, the assumption is that for the file in consideration, the index is missing in .gd.

Local	Remote	MD5 Matches	Action
Present	Present	Yes	Fetch metadata from GDrive, and create the index.
Present	Present	No	Compare times. Inform the user which copy is newer. Fetch metadata from GDrive, and create the index.
Present	Absent	No	Do nothing.
Absent	Present	No	Do nothing.

Currently, the only way to get the indices to become sane again appears to be a full pull (or push as appropriate). That, however, could be very inconvenient, should the directories involved be large.

[odeke-em]

Thanks @js-ojus for reporting this and for this suggestion. Great idea, I started working on one about three hours ago but I had to go out. I'll keep plugging at it so please watch this space.

[odeke-em]

Oh btw, if pulling is tedious there are options that allows you to exclude certain operations e.g
--no-clobber or --exclude-ops delete etc.

[js-ojus]

It is a pleasure working with you, @odeke-em! You are so responsive!

[odeke-em]

Thank you for the kind words, it is a pleasure working with you too.

[odeke-em]

Hello @js-ojus sorry for the late reply, I was occupied yesterday hanging out with a friend who is immigrating to another country but I got sometime this morning and I have implemented a PR here: #256. It contains sample usage. Am off to bed now but will be up in 5 hours or so and will merge it in.
Thank you for your patience.

[odeke-em]

Hello @js-ojus, please get the latest from master. This issue was addressed by PR #256. Sample usage in the README as well as that PR.
Btw in regards to fetching indices when modification dates changed, thing is re-indexing in that scenario creates a false no change needed result since differences in index files and remote are necessary to enable pulling and pushing content.
Thank you for reporting this issue as well as for the suggestions. Please re-open if persists.

[js-ojus]

Thanks so much @odeke-em! I shall try this, and let you know the outcome.

[js-ojus]

Creation of indices has taken place as intended. Good thus far!

Now, running drive push continues to mark the same set of files for a re-push. I have investigated it a little. The reason is that the 'mtime' on GDrive is set to whatever was the UTC at the time of the original push, not that of the pushed file itself.

To illustrate using an example, the local file has 'mtime' 2007-11-08 18:30:02, while the corresponding copy in GDrive has 2015-06-15 17:32:56 (the time when push uploaded this file).

For those files whose indices were created properly upon the first push, the local 'mtime' and the remote one match perfectly.

It will be good if push and pull gained a capability (sub-command) to update the target's (remote file's metadata in the case of push, local file's index in the case of pull) 'mtime' when MD5 sums match.

Alternatively, such a sub-command could exist for index itself. E.g. drive index push and drive index pull.

The overriding criteria here is that MD5 sums match, but 'mtime's do not! Other cases are already handled quite well.

Should you think that it is reasonable, I can create an issue for the same.

[odeke-em]

Hello @js-ojus, thanks for the investigation. I think this feedback is more related to the issue that spun this request than to this issue itself. If we could move the discussion there, that'd be great.

Unless some bug just crept, drive always updates the mtime derived from local file itself and this has been working since I made this fork, you can even try this out yourself by moving in and old file, pushing it upstream, making sure there are no more changes, then doing a drive touch <remote_file>, inspecting with your browser, as well as stating and diffing. Then re-run drive push <remote_file> and the older date that was present locally is re-updated. Also at the end of every pull or push index files are updated.
Please let me know what you think.