New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

suggestion for megacopy.exe : --overwrite-if-different-size #274

Closed
aaz1887 opened this Issue Mar 27, 2017 · 12 comments

Comments

7 participants
@aaz1887

aaz1887 commented Mar 27, 2017

I suggest you to make an additional switch to megacopy.exe, such as --overwrite-if-different-size
for upload a file with equal name if file-size in mega-cloud is different.

Excuse my poor English. Thank you very much

@jiggunjer

This comment has been minimized.

Show comment
Hide comment
@jiggunjer

jiggunjer Mar 29, 2017

Seconded, though I wouldn't recommend filesize; maybe md5? Or just a dumb --overwrite switch. Current behavior is to give a non-terminating error if the target file/folder exists, making it impossible to update text files that change with megaput/megacopy.

My fix, for the time being, is to remove all duplicate files/folders first, then upload from the newer source. This is not perfect because sometimes files aren't changed but they still get replaced--making it a dumb overwrite.

jiggunjer commented Mar 29, 2017

Seconded, though I wouldn't recommend filesize; maybe md5? Or just a dumb --overwrite switch. Current behavior is to give a non-terminating error if the target file/folder exists, making it impossible to update text files that change with megaput/megacopy.

My fix, for the time being, is to remove all duplicate files/folders first, then upload from the newer source. This is not perfect because sometimes files aren't changed but they still get replaced--making it a dumb overwrite.

@i30817

This comment has been minimized.

Show comment
Hide comment
@i30817

i30817 Jun 17, 2017

I'd like to go further and have a option to delete files which exist in the local folder but not the cloud one. This is because many people share public folders as 'updateable' and delete files there, and i want the last version locally not 'the last and what existed before'. I thought megasync was able to do this and was unpleasantly surprised when it didn't.

i30817 commented Jun 17, 2017

I'd like to go further and have a option to delete files which exist in the local folder but not the cloud one. This is because many people share public folders as 'updateable' and delete files there, and i want the last version locally not 'the last and what existed before'. I thought megasync was able to do this and was unpleasantly surprised when it didn't.

@groundstack

This comment has been minimized.

Show comment
Hide comment
@groundstack

groundstack Jul 3, 2017

This would be a nice feature. Right now as far as I know it just skips modified files so you are never sure if you have the latest version on mega.nz. Is there a workaround that does not involve re-uploading all existing files? Thanks!

groundstack commented Jul 3, 2017

This would be a nice feature. Right now as far as I know it just skips modified files so you are never sure if you have the latest version on mega.nz. Is there a workaround that does not involve re-uploading all existing files? Thanks!

@megous megous added the enhancement label Jul 22, 2017

@penright

This comment has been minimized.

Show comment
Hide comment
@penright

penright Jul 27, 2017

It might be easier just to ask for rsync capabilities.
On second thought, can we have rsync capabilities? :-)
Might be easier just to make a new tool, megarsync
Does mega have a md5 function or something like it? That way you don't have to pull down the file to calculate if needed to be updated.

penright commented Jul 27, 2017

It might be easier just to ask for rsync capabilities.
On second thought, can we have rsync capabilities? :-)
Might be easier just to make a new tool, megarsync
Does mega have a md5 function or something like it? That way you don't have to pull down the file to calculate if needed to be updated.

@megous

This comment has been minimized.

Show comment
Hide comment
@megous

megous Jul 27, 2017

Owner

No md5. Mega doesn't even have a proper timestamping for uploaded files (there's "ts" field, but it only stores time of upload, not a modification time of the original file, so it's useless for comparison). It is possible to store whatever extra metadata along the filesystem nodes, but it would only work with megatools and would not be compatible with any other tools/mega.nz website.

At least that was the state when I first wrote megatools.

So yes, it's doable, but it would be very limited or incompatible with other tools. If you need sync, you can probably use MEGAsync.

Owner

megous commented Jul 27, 2017

No md5. Mega doesn't even have a proper timestamping for uploaded files (there's "ts" field, but it only stores time of upload, not a modification time of the original file, so it's useless for comparison). It is possible to store whatever extra metadata along the filesystem nodes, but it would only work with megatools and would not be compatible with any other tools/mega.nz website.

At least that was the state when I first wrote megatools.

So yes, it's doable, but it would be very limited or incompatible with other tools. If you need sync, you can probably use MEGAsync.

@i30817

This comment has been minimized.

Show comment
Hide comment
@i30817

i30817 Jul 27, 2017

I haven't tried megacopy yet too much to see if it's different, but i think megasync actually sucks at non-private use.

This is because you need to copy a public folder into your own account before you can 'megasync' it, and therefore you need to copy it every time you want to check updates on a public folder. This stinks and i can't help but think it's on purpose.

Another problem is that i can't specify that i don't want deleting locally to not reflect on the server, or having a 'extra' file that was deleted remotely not be uploaded, or be deleted (especially problematic when updating the copy).

So for these reasons, i agree that a 'rsync' functionality from the cmd line would blow megasync out of the water, even without the bi-directionality that it offers. Indeed, it could be said to be filling a hole in the feature set in not having that bi-directionality and offering a mirror feature for public links which the client doesn't own.

i30817 commented Jul 27, 2017

I haven't tried megacopy yet too much to see if it's different, but i think megasync actually sucks at non-private use.

This is because you need to copy a public folder into your own account before you can 'megasync' it, and therefore you need to copy it every time you want to check updates on a public folder. This stinks and i can't help but think it's on purpose.

Another problem is that i can't specify that i don't want deleting locally to not reflect on the server, or having a 'extra' file that was deleted remotely not be uploaded, or be deleted (especially problematic when updating the copy).

So for these reasons, i agree that a 'rsync' functionality from the cmd line would blow megasync out of the water, even without the bi-directionality that it offers. Indeed, it could be said to be filling a hole in the feature set in not having that bi-directionality and offering a mirror feature for public links which the client doesn't own.

@i30817

This comment has been minimized.

Show comment
Hide comment
@i30817

i30817 Jul 27, 2017

Basically, i'd appreciate very much even a simple 'rmlocal' tool that would delete files on a dir not on the remote dir (just those with different names), before using MegaSync even if you don't want to implement the 'this is a different file now and needs a update' functionality that MegaSync does, since this is a use case they 'forgot' and it's a pain in the ass to do manually.

I'd prefer both ofc, because MegaSync doesn't 'do' public folders and copying to the account before doing this is tiresome (copy, megals dir, delete extra files not on list, sync folders), and the alternatives (JDownload etc) don't have the code to skip useless downloads.

i30817 commented Jul 27, 2017

Basically, i'd appreciate very much even a simple 'rmlocal' tool that would delete files on a dir not on the remote dir (just those with different names), before using MegaSync even if you don't want to implement the 'this is a different file now and needs a update' functionality that MegaSync does, since this is a use case they 'forgot' and it's a pain in the ass to do manually.

I'd prefer both ofc, because MegaSync doesn't 'do' public folders and copying to the account before doing this is tiresome (copy, megals dir, delete extra files not on list, sync folders), and the alternatives (JDownload etc) don't have the code to skip useless downloads.

@megous megous self-assigned this Jul 27, 2017

@megous megous added this to the stable milestone Jul 27, 2017

@i30817

This comment has been minimized.

Show comment
Hide comment
@i30817

i30817 Jul 28, 2017

edited: a better, safer version of the same thing now on a gist:

BTW, if anyone's interested, currently i'm doing this to sync public folders and trash the extra local files:

  1. import the the public folder to your cloud with the same name, deleting previous version.
  2. download and use this script like so: megaprune <email> <password> <local_dir> <remote_dir>. Remote dir doesn't need the same name as the local dir.

This finds all files on the local dir and filters out those that aren't in the cloud (by filename, not hash) and puts them in trash. <remote_dir> is the bare name no /Root. You need megals and realpath (which may or not be on your linux).

Then i use megasync for the affected folder to download the files with conflicting hashes. A warning about megasync: i've noticed after doing this, that it sometimes 'uploads' a 0 bytes bogus file. What i think it's happening is a download failing, but the 'preliminary' 0 byte file being created on the local folder, and then it being uploaded. You can 'fix' this by deleting locally the bogus file and going into your cloud mega trashcan and moving the (right size) deleted file again to the directory it was. It'll redownload.

i30817 commented Jul 28, 2017

edited: a better, safer version of the same thing now on a gist:

BTW, if anyone's interested, currently i'm doing this to sync public folders and trash the extra local files:

  1. import the the public folder to your cloud with the same name, deleting previous version.
  2. download and use this script like so: megaprune <email> <password> <local_dir> <remote_dir>. Remote dir doesn't need the same name as the local dir.

This finds all files on the local dir and filters out those that aren't in the cloud (by filename, not hash) and puts them in trash. <remote_dir> is the bare name no /Root. You need megals and realpath (which may or not be on your linux).

Then i use megasync for the affected folder to download the files with conflicting hashes. A warning about megasync: i've noticed after doing this, that it sometimes 'uploads' a 0 bytes bogus file. What i think it's happening is a download failing, but the 'preliminary' 0 byte file being created on the local folder, and then it being uploaded. You can 'fix' this by deleting locally the bogus file and going into your cloud mega trashcan and moving the (right size) deleted file again to the directory it was. It'll redownload.

@armandg

This comment has been minimized.

Show comment
Hide comment
@armandg

armandg Apr 26, 2018

How about implementing same feature that youtube-dl has, the --continue flag.

-c, --continue
              Force resume of partially downloaded files.  By default, youtube-dl
              will resume downloads if possible.

To be fair, I have no idea how to implement this, just throwing it out there.

armandg commented Apr 26, 2018

How about implementing same feature that youtube-dl has, the --continue flag.

-c, --continue
              Force resume of partially downloaded files.  By default, youtube-dl
              will resume downloads if possible.

To be fair, I have no idea how to implement this, just throwing it out there.

@megous

This comment has been minimized.

Show comment
Hide comment
@megous

megous Apr 26, 2018

Owner

@armandg That is already implemented (without an option to disable it), also this thread is about something else.

Owner

megous commented Apr 26, 2018

@armandg That is already implemented (without an option to disable it), also this thread is about something else.

@armandg

This comment has been minimized.

Show comment
Hide comment
@armandg

armandg Apr 26, 2018

@megous I am obviously out of my element, and will disappear from this thread.

armandg commented Apr 26, 2018

@megous I am obviously out of my element, and will disappear from this thread.

@megous

This comment has been minimized.

Show comment
Hide comment
@megous

megous Jul 31, 2018

Owner

If sync gets implemented, this will be part of it. So let's move this to #20

Owner

megous commented Jul 31, 2018

If sync gets implemented, this will be part of it. So let's move this to #20

@megous megous closed this Jul 31, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment