Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AzCopy Sync Blob to Blob support #116

Closed
thomasmaurer opened this issue Nov 6, 2018 · 6 comments
Labels

Comments

@thomasmaurer
Copy link
Contributor

@thomasmaurer thomasmaurer commented Nov 6, 2018

Which version of the AzCopy was used?

Note: The version is visible when running AzCopy without any argument

10.0

Which platform are you using? (ex: Windows, Mac, Linux)

Windows

What problem was encountered?

Azcopy sync now supports file system to blob, it should also support blob to blob sync.

@Kapanther

This comment has been minimized.

Copy link

@Kapanther Kapanther commented Feb 11, 2019

@zezha-msft any news on blob to blob sync working? is this still an enhancement

@zezha-msft

This comment has been minimized.

Copy link
Member

@zezha-msft zezha-msft commented Feb 12, 2019

Hi @Kapanther, I'm currently working on a change that would enable us to add this functionality more easily. I'll update this thread once I have a better idea of the timeline.

@Kapanther

This comment has been minimized.

Copy link

@Kapanther Kapanther commented Apr 9, 2019

@zezha-msft I see blob-blob support has been added in 10.0.9? If so i'll go test for you.

@zezha-msft

This comment has been minimized.

Copy link
Member

@zezha-msft zezha-msft commented Apr 9, 2019

Hi @Kapanther, not yet for sync, it's been added for copy, to transfer between accounts all types of blobs.

@JohnRusk

This comment has been minimized.

Copy link
Member

@JohnRusk JohnRusk commented May 27, 2019

BTW, if the changes to the source location are purely additive (i.e. new files added, but no updates to existing files) then as a workaround it should work to use copy with --overwrite=false. I haven't done any tests focused on using this as an interim replacement for blob-to-blob sync, but I can't see any reason why it wouldn't work as long as you know for sure that source files are only added (never updated).

To be clear on what happens if you use copy with overwrite=false: AzCopy scans the names of all the source files, but it only transfers those which don't already exist at the destination. With "copy" and overwrite=false, the check is purely based on whether the file exists, there is no check of last modified dates. (There is also no option to delete destination files that no longer exist at the source).

(Sync, for blob-to-blob, will check the last modified dates and will allow optional destination-side deletion of files that have been removed from the source.)

@JohnRusk

This comment has been minimized.

Copy link
Member

@JohnRusk JohnRusk commented Nov 15, 2019

Blob to Blob sync was release in version 10.3. See https://docs.microsoft.com/en-us/azure/storage/common/storage-ref-azcopy-sync

@JohnRusk JohnRusk closed this Nov 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.