Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(heavier_once_cell): take_and_deinit should take ownership #7185

Merged
merged 2 commits into from
Mar 20, 2024

Conversation

koivunej
Copy link
Member

@koivunej koivunej commented Mar 20, 2024

Small fix to remove confusing mut bindings.

Builds upon #7175, split off from #7030. Cc: #5331.

Copy link

github-actions bot commented Mar 20, 2024

2706 tests run: 2573 passed, 0 failed, 133 skipped (full report)


Flaky tests (1)

Postgres 16

  • test_backward_compatibility: release

Code coverage* (full report)

  • functions: 28.3% (7135 of 25210 functions)
  • lines: 46.9% (43716 of 93293 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
4c55a8f at 2024-03-20T19:41:25.609Z :recycle:

@koivunej koivunej force-pushed the joonas/layer-always-init-on-download branch from 0b99baf to ecbddfd Compare March 20, 2024 16:05
@koivunej koivunej force-pushed the joonas/heavier-once-cell-small-fix branch from 0d27d6c to 86c3d10 Compare March 20, 2024 16:24
@koivunej koivunej marked this pull request as ready for review March 20, 2024 18:03
@koivunej koivunej requested a review from a team as a code owner March 20, 2024 18:03
@koivunej koivunej requested review from VladLazar and removed request for a team March 20, 2024 18:03
Base automatically changed from joonas/layer-always-init-on-download to main March 20, 2024 18:37
@koivunej koivunej force-pushed the joonas/heavier-once-cell-small-fix branch from 86c3d10 to 4c55a8f Compare March 20, 2024 18:38
@koivunej koivunej enabled auto-merge (squash) March 20, 2024 20:03
@koivunej koivunej disabled auto-merge March 20, 2024 20:31
@koivunej koivunej merged commit a95c41f into main Mar 20, 2024
53 checks passed
@koivunej koivunej deleted the joonas/heavier-once-cell-small-fix branch March 20, 2024 22:42
koivunej added a commit that referenced this pull request Mar 21, 2024
## Problem

The current implementation of struct Layer supports canceled read
requests, but those will leave the internal state such that a following
`Layer::keep_resident` call will need to repair the state. In
pathological cases seen during generation numbers resetting in staging
or with too many in-progress on-demand downloads, this repair activity
will need to wait for the download to complete, which stalls disk
usage-based eviction. Similar stalls have been observed in staging near
disk-full situations, where downloads failed because the disk was full.

Fixes #6028 or the "layer is present on filesystem but not evictable"
problems by:
1. not canceling pending evictions by a canceled
`LayerInner::get_or_maybe_download`
2. completing post-download initialization of the `LayerInner::inner`
from the download task

Not canceling evictions above case (1) and always initializing (2) lead
to plain `LayerInner::inner` always having the up-to-date information,
which leads to the old `Layer::keep_resident` never having to wait for
downloads to complete. Finally, the `Layer::keep_resident` is replaced
with `Layer::is_likely_resident`. These fix #7145.

## Summary of changes

- add a new test showing that a canceled get_or_maybe_download should
not cancel the eviction
- switch to using a `watch` internally rather than a `broadcast` to
avoid hanging eviction while a download is ongoing
- doc changes for new semantics and cleanup
- fix `Layer::keep_resident` to use just `self.0.inner.get()` as truth
as `Layer::is_likely_resident`
- remove `LayerInner::wanted_evicted` boolean as no longer needed

Builds upon: #7185. Cc: #5331.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants