Checksum did not change when the download file failed to retry

[wwjjkk]

Describe the bug
Checksum did not change when the download file failed to redownload

To Reproduce
When downloading a file, overwriting the file will cause the download to fail
Information is as follows:

[Warn] [Thread 0] Attempt 0 - Failure occurred while downloading http://xxx/artifactory/xxx.txt - Checksum mismatch for /work/xxx.txt, expected: 716349e89d1260a1f8129520ec69953738e818c3, actual: 0caf2f7aec8045d47744520fd1cc942f5fd5b08d
[Warn] [Thread 0] Attempt 1 - Failure occurred while downloading http://xxx/artifactory/xxx.txt - Checksum mismatch for /work/xxx.txt, expected: 716349e89d1260a1f8129520ec69953738e818c3, actual: 0caf2f7aec8045d47744520fd1cc942f5fd5b08d
[Warn] [Thread 0] Attempt 2 - Failure occurred while downloading http://xxx/artifactory/xxx.txt - Checksum mismatch for /work/xxx.txt, expected: 716349e89d1260a1f8129520ec69953738e818c3, actual: 0caf2f7aec8045d47744520fd1cc942f5fd5b08d
[Warn] [Thread 0] Attempt 3 - Failure occurred while downloading http://xxx/artifactory/xxx.txt - Checksum mismatch for /work/xxx.txt, expected: 716349e89d1260a1f8129520ec69953738e818c3, actual: 0caf2f7aec8045d47744520fd1cc942f5fd5b08d
[Error] [Thread 0]  Received an error: Checksum mismatch for /work/xxx.txt, expected: 716349e89d1260a1f8129520ec69953738e818c3, actual: 0caf2f7aec8045d47744520fd1cc942f5fd5b08d

In our business, we need to recover some download files frequently, which is easy to happen when downloading, This will cause pipeline execution to fail

Expected behavior
Checksum change when the download file failed to redownload.Make sure the download works when we retry

[eyalbe4]

@wwjjkk,
JFrog CLI calculates the checksum after the file is downloaded. It then compares it with the checksum received from Artifactory. We need to figure out why the checksums are different, or in other words, which checksum is the correct one, the one received from Artifactory or the one calculated by JFrog CLI. I suggest that you calculate the SHA1 after the file is downloaded to figure this out. If JFrog CLI is right, the next step would be figuring out what happened on Artifactory's side.

[wwjjkk]

@wwjjkk,
JFrog CLI calculates the checksum after the file is downloaded. It then compares it with the checksum received from Artifactory. We need to figure out why the checksums are different, or in other words, which checksum is the correct one, the one received from Artifactory or the one calculated by JFrog CLI. I suggest that you calculate the SHA1 after the file is downloaded to figure this out. If JFrog CLI is right, the next step would be figuring out what happened on Artifactory's side.

It occurs during the download of a file and the file is overwritten.But what is not understood is why automatic retries still fail.Is there a problem with the retry mechanism？

Suppose that there are two nodes A and B in the cluster. We first uploaded to node A, then downloading from node A. at this time, upload and rewrite the file through from node B. as local cache is preferred, will it lead to the above situation？

[wwjjkk]

@eyalbe4
Through the debug information in the logs, I found that through "jfrog rt dl xxx/xxx/" download directory , JFrog CLI would first query all the file information with AQL,then calculates the checksum after the file is downloaded.So when downloading, if the file is overwritten, the validation must fail.

Can you modify the download logic to retrieve validation information if validation fails？Or allow the user to specify whether or not to enable this logic by argument

[github-actions]

This issue has been marked as stale due to 6 months of inactivity. As part of our effort to address every issue properly, please feel free to remove the stale label or keep this issue active by leaving a comment. Otherwise, it will be closed in 7 days

[github-actions]

This issue was closed due to 7 days of inactivity after being marked as stale. Feel free to reopen it if it remains relevant.