Skip to content

fix: wait for stretto cache insert before validation #2215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sanity merged 1 commit into from
Dec 5, 2025
+9 −0
Merged

fix: wait for stretto cache insert before validation #2215

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions crates/core/src/wasm_runtime/contract_store.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,11 @@ impl ContractStore {
let data = code.data().to_vec();
self.contract_cache
.insert(*code_hash, Arc::new(ContractCode::from(data)), size);
// Wait for the cache insert to be visible. Stretto uses background threads
// for inserts, and without wait() the value may not be immediately visible
// to subsequent get() calls (eventual consistency). This is critical because
// validate_state() needs to fetch the contract immediately after storing.
let _ = self.contract_cache.wait();
Copy link
Collaborator Author

@sanity sanity Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@claude What thread will this block?

Copy link

@claude claude bot Dec 5, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude finished @sanity's task —— View job

Analysis

The cache.wait() call blocks the current tokio worker thread that's executing the async function.

Call chain:

  1. store_contract() is called from verify_and_store_contract() (runtime.rs:1092)
  2. verify_and_store_contract() is an async fn running on the tokio async runtime
  3. When the tokio executor schedules this task on a worker thread, that worker thread executes the code
  4. The cache.wait() call is a synchronous blocking operation (not async)
  5. This blocks the tokio worker thread until the stretto background threads complete the buffered insert

Impact:

  • While one tokio worker thread is blocked waiting for the cache, other tokio worker threads can still process other async tasks
  • The blocking duration should be minimal (just waiting for the cache insert buffer to flush)
  • However, this is technically blocking async code, which is generally discouraged in tokio

Alternative approach:
If stretto provided an async wait() method (like async fn wait() or returning a future), that would be preferable as it would allow the tokio worker thread to handle other tasks while waiting. But since stretto's Cache is synchronous (not AsyncCache), and the wait time should be very short (just flushing the insert buffer), the current approach is acceptable.

Note: The code uses stretto::Cache (sync), not stretto::AsyncCache. The crate does have both variants available (Cargo.toml enables both "async" and "sync" features).

Copy link
Collaborator Author

@sanity sanity Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify: while this technically blocks the tokio task, the important context is that this is not a cross-request race condition.

The store_contract() → validate_state() sequence happens within a single PUT request, synchronously. Looking at verify_and_store_contract() in runtime.rs:

// Line 1092: store the contract
self.runtime.contract_store.store_contract(contract)?;

// Line 1105: immediately validate (needs to fetch contract)
let result = self.runtime.validate_state(...)?;

The validate_state() call immediately fetches the contract we just stored via prepare_contract_call() → fetch_contract(). Without wait(), the buffered insert hasn't been processed yet, so get() returns None and we get "contract not found".

The window isn't "narrow" - it's essentially guaranteed to fail because it's two consecutive function calls in the same thread, and stretto buffers the insert for background processing.

[AI-assisted - Claude]


// save on disc
let version = APIVersion::from(contract);
Expand Down
4 changes: 4 additions & 0 deletions crates/core/src/wasm_runtime/delegate_store.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,10 @@ impl DelegateStore {
let code_size = data.len() as i64;
self.delegate_cache
.insert(*code_hash, delegate.code().clone().into_owned(), code_size);
// Wait for the cache insert to be visible. Stretto uses background threads
// for inserts, and without wait() the value may not be immediately visible
// to subsequent get() calls (eventual consistency).
let _ = self.delegate_cache.wait();

// save on disc
let version = APIVersion::from(delegate.clone());
Expand Down
Loading