Skip to content

cleanup: don't do stale check when passing --prune.#10189

Merged
MikeMcQuaid merged 1 commit intoHomebrew:masterfrom
MikeMcQuaid:cleanup_faster_prune
Dec 31, 2020
Merged

cleanup: don't do stale check when passing --prune.#10189
MikeMcQuaid merged 1 commit intoHomebrew:masterfrom
MikeMcQuaid:cleanup_faster_prune

Conversation

@MikeMcQuaid
Copy link
Copy Markdown
Member

This speeds up --prune fairly considerably.

Before:

$ hyperfine --warmup 3 'brew cleanup --prune=365'
Benchmark #1: brew cleanup --prune=365
  Time (mean ± σ):     33.352 s ±  0.654 s    [User: 21.684 s, System: 6.458 s]
  Range (min … max):   31.901 s … 34.096 s    10 runs

After:

$ hyperfine --warmup 3 'brew cleanup --prune=365'
Benchmark #1: brew cleanup --prune=365
  Time (mean ± σ):      6.821 s ±  0.186 s    [User: 2.225 s, System: 3.926 s]
  Range (min … max):    6.578 s …  7.178 s    10 runs

Closes #10177
Fixes #10172

This speeds up `--prune` fairly considerably.

Before:
```
$ hyperfine --warmup 3 'brew cleanup --prune=365'
Benchmark #1: brew cleanup --prune=365
  Time (mean ± σ):     33.352 s ±  0.654 s    [User: 21.684 s, System: 6.458 s]
  Range (min … max):   31.901 s … 34.096 s    10 runs
```

After:
```
$ hyperfine --warmup 3 'brew cleanup --prune=365'
Benchmark #1: brew cleanup --prune=365
  Time (mean ± σ):      6.821 s ±  0.186 s    [User: 2.225 s, System: 3.926 s]
  Range (min … max):    6.578 s …  7.178 s    10 runs
```

Closes #10177
Fixes #10172
@BrewTestBot
Copy link
Copy Markdown
Contributor

Review period will end on 2021-01-01 at 14:00:34 UTC.

@BrewTestBot BrewTestBot added the waiting for feedback Merging is blocked until sufficient time has passed for review label Dec 31, 2020
@MikeMcQuaid MikeMcQuaid added the critical Critical change which should be shipped as soon as possible. label Dec 31, 2020
@BrewTestBot BrewTestBot removed the waiting for feedback Merging is blocked until sufficient time has passed for review label Dec 31, 2020
@BrewTestBot
Copy link
Copy Markdown
Contributor

Review period skipped due to critical label.

@MikeMcQuaid MikeMcQuaid merged commit a2594df into Homebrew:master Dec 31, 2020
@MikeMcQuaid MikeMcQuaid deleted the cleanup_faster_prune branch December 31, 2020 16:16
@fxcoudert
Copy link
Copy Markdown
Member

fxcoudert commented Dec 31, 2020

With that merged, the timings for Intel Big Sur runners have gone from:

  • cleanup-before 2m50s
  • cleanup-after 3m0s

to

  • cleanup-before 45s
  • cleanup-after 1m20s

I'd have to check what is still taking so long. Even with thousands of files, getting mtime and ctime should not take an entire minute. For reference, even on the ARM runner that has the most cached files, it's taking 5 seconds in the worst case.

@MikeMcQuaid
Copy link
Copy Markdown
Member Author

I'd have to check what is still taking so long. Even with thousands of files, getting mtime and ctime should not take an entire minute. For reference, even on the ARM runner that has the most cached files, it's taking 5 seconds in the worst case.

I don't think the ARM runner is a good comparison for various reasons:

  • it's a new architecture that's been custom-designed to run macOS
  • it's not running in a VM
  • it's using SSD storage

find -delete could be an acceptable replacement to put in brew test-bot itself instead of brew cleanup --prune.

@BrewTestBot BrewTestBot added the outdated PR was locked due to age label Jan 31, 2021
@Homebrew Homebrew locked as resolved and limited conversation to collaborators Jan 31, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

critical Critical change which should be shipped as soon as possible. outdated PR was locked due to age

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI is too slow because of brew cleanup --prune=3

3 participants