AzCopy Sync Blob to Blob support

[thomasmaurer]

Which version of the AzCopy was used?

Note: The version is visible when running AzCopy without any argument

10.0

Which platform are you using? (ex: Windows, Mac, Linux)

Windows

What problem was encountered?

Azcopy sync now supports file system to blob, it should also support blob to blob sync.

[Kapanther]

@zezha-msft any news on blob to blob sync working? is this still an enhancement

[zezha-msft]

Hi @Kapanther, I'm currently working on a change that would enable us to add this functionality more easily. I'll update this thread once I have a better idea of the timeline.

[Kapanther]

@zezha-msft I see blob-blob support has been added in 10.0.9? If so i'll go test for you.

[zezha-msft]

Hi @Kapanther, not yet for sync, it's been added for copy, to transfer between accounts all types of blobs.

[JohnRusk]

BTW, if the changes to the source location are purely additive (i.e. new files added, but no updates to existing files) then as a workaround it should work to use copy with --overwrite=false. I haven't done any tests focused on using this as an interim replacement for blob-to-blob sync, but I can't see any reason why it wouldn't work as long as you know for sure that source files are only added (never updated).

To be clear on what happens if you use copy with overwrite=false: AzCopy scans the names of all the source files, but it only transfers those which don't already exist at the destination. With "copy" and overwrite=false, the check is purely based on whether the file exists, there is no check of last modified dates. (There is also no option to delete destination files that no longer exist at the source).

(Sync, for blob-to-blob, will check the last modified dates and will allow optional destination-side deletion of files that have been removed from the source.)

[JohnRusk]

Blob to Blob sync was release in version 10.3. See https://docs.microsoft.com/en-us/azure/storage/common/storage-ref-azcopy-sync