az storage fs directory download replaces local azcopy binary with old version

[jakevc]

Describe the bug

When running az storage fs directory download command to download a file from an Azure storage account with ADLS Gen2 enabled, the command seems to download azcopy version 10.13.0, overwriting any local version of the binary you have.

Related command

az storage fs directory download --recursive

Errors

No Errors

Issue script & Debug output

Start version

$ azcopy --version
azcopy version 10.20.1
INFO: azcopy 10.20.1: A newer version 10.21.0 is available to download

Download command

$ az storage fs directory download -f snaketest --account-name eacriprodpipelines -d snaketest --recursive
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus

There are no credentials provided in your command and environment, we will query for account key for your storage account.
It is recommended to provide --connection-string, --account-key or --sas-token in your command as credentials.

In addition, setting the corresponding environment variables can avoid inputting credentials in your command. Please use --help to get more information about environment variable usage.
Azcopy command: ['/home/********/bin/azcopy', 'copy', 'https://*********.dfs.core.windows.net/snaketest***, 'snaketest', '--recursive']
INFO: Scanning...
INFO: Any empty folders will be processed, because source and destination both support folders
INFO: azcopy: A newer version 10.21.0 is available to download


Job ced81976-7ab6-e249-7ac4-9f237a51ef72 has started
Log file is located at: /home/*********/.azcopy/ced81976-7ab6-e249-7ac4-9f237a51ef72.log

0.0 %, 0 Done, 0 Failed, 20 Pending, 0 Skipped, 20 Total,


Job ced81976-7ab6-e249-7ac4-9f237a51ef72 summary
Elapsed Time (Minutes): 0.0334
Number of File Transfers: 16
Number of Folder Property Transfers: 4
Total Number of Transfers: 20
Number of Transfers Completed: 20
Number of Transfers Failed: 0
Number of Transfers Skipped: 0
TotalBytesTransferred: 18095520

$ azcopy --version
azcopy version 10.13.0
INFO: azcopy: A newer version 10.21.0 is available to download

The storage command seems to have overwritten by binary with an older one!

Expected behavior

The azure storage fs directory command doesn't downgrade my azcopy version!

Environment Summary

az --version
azure-cli                         2.53.0

core                              2.53.0
telemetry                          1.1.0

Extensions:
interactive                        0.5.3
storage-preview                  1.0.0b1

Dependencies:
msal                            1.24.0b2
azure-mgmt-resource             23.1.0b2

Python location '/usr/bin/python3.9'
Extensions directory '/home/******/.azure/cliextensions'

Python (Linux) 3.9.16 (main, May 31 2023, 12:21:58)
[GCC 8.5.0 20210514 (Red Hat 8.5.0-18)]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[slagelwa]

I seem to have also been bitten by this. I updated to 10.21.0 about a week or so ago and don't remember running any az storage commands, but was really surprised when azcopy --version suddenly started reporting 10.13.0. It was even more confusing since I named the executable azcopy.10.21.0 and had an azcopy link pointing to it.

[jakevc]

Nice! Also might be nice to bump the AZCOPY_VERSION = '10.13.0', but hard to say if that would break other subcommands that download azcopy. That being said, this change removes the enforcement of (tested) compatibility in these sub commands with a pinned version of azcopy, potentially leading to other downstream bugs...

[slagelwa]

Curious -- is it really a good idea to automatically install a program behind the scenes?