Skip to content

Commit

Permalink
apacheGH-41243: [Release][Packaging] Avoid needless download by "arch…
Browse files Browse the repository at this point in the history
…ery crossbow download-artifacts" (apache#41244)

### Rationale for this change

We don't need to re-download already downloaded artifacts.

### What changes are included in this PR?

Check local path and skip download if there is downloaded artifact on local.
 
### Are these changes tested?

Yes. I've tested this with 16.0.0 RC0 release.

### Are there any user-facing changes?

No.
* GitHub Issue: apache#41243

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
  • Loading branch information
kou authored and vibhatha committed May 25, 2024
1 parent 764ee42 commit 0f6f229
Showing 1 changed file with 11 additions and 1 deletion.
12 changes: 11 additions & 1 deletion dev/archery/archery/crossbow/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -505,7 +505,17 @@ def asset_callback(task_name, task, asset):
if asset is not None:
path = target_dir / task_name / asset.name
path.parent.mkdir(exist_ok=True)
if not dry_run:

def need_download():
if dry_run:
return False
if not path.exists():
return True
if path.stat().st_size != asset.size:
return True
return False

if need_download():
import github3
max_n_retries = 5
n_retries = 0
Expand Down

0 comments on commit 0f6f229

Please sign in to comment.