Skip to content

Reduce DataVolume wait_deleted timeout to 1 minute #2292

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Reduce DataVolume wait_deleted timeout to 1 minute #2292

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ocp_resources/datavolume.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ def to_dict(self) -> None:
f"{self.api_group}/storage.deleteAfterCompletion": (self.delete_after_completion)
})

def wait_deleted(self, timeout=TIMEOUT_4MINUTES):
def wait_deleted(self, timeout=TIMEOUT_1MINUTE):
Copy link

@coderabbitai coderabbitai bot Jan 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Revert the timeout reduction or provide evidence for 1-minute sufficiency

The 1-minute timeout for DataVolume deletion appears risky and inconsistent with the codebase patterns:

  1. Most resources use 4-minute timeouts for deletion operations
  2. DataVolume deletion requires two sequential operations (DV + PVC deletion)
  3. Each operation would effectively get only ~30 seconds
  4. No evidence in codebase supporting 1-minute being sufficient

Please either:

  • Revert to TIMEOUT_4MINUTES for consistency and reliability
  • Provide metrics/data showing that 1-minute consistently suffices for both operations
🔗 Analysis chain

Verify that 1-minute timeout is sufficient for DataVolume deletion.

Reducing the timeout from 4 minutes to 1 minute for both DataVolume and PVC deletion could be risky:

  1. The timeout is shared between two sequential operations (DV deletion + PVC deletion)
  2. Each operation effectively gets only ~30 seconds
  3. This might not be sufficient in slower environments or under heavy load

Please provide data/evidence showing that 1 minute is consistently sufficient for both operations to complete.

Run this script to analyze historical deletion times:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Analyze DataVolume deletion times from logs to verify if 1-minute timeout is sufficient

# Search for DataVolume deletion logs with timestamps
rg --multiline --multiline-dotall 'Wait DV.*deletion.*\n.*deletion.*completed' -A 5

# Search for any timeout errors related to DataVolume deletion
rg 'TimeoutExpiredError.*deletion.*DataVolume'

Length of output: 134

Script:

#!/bin/bash
# Search for test files and code related to DataVolume deletion timeouts

# Find test files related to DataVolume deletion
echo "=== Test files with DataVolume deletion tests ==="
rg -l "test.*delete.*datavolume" --type python
rg -l "delete.*test.*datavolume" --type python

# Look for timeout constants and their usage
echo -e "\n=== Timeout constants and their usage ==="
rg "TIMEOUT_[1234]MINUTE" --type python -B 1 -A 1

# Look for any DataVolume deletion related failures in test files
echo -e "\n=== DataVolume deletion failure patterns in tests ==="
rg "deletion.*fail" --type python -A 2

Length of output: 645

Script:

#!/bin/bash
# Search for DataVolume deletion and timeout patterns without file type filtering

# Find files with DataVolume deletion tests
echo "=== DataVolume deletion related files ==="
rg -l "def test.*delete.*[Dd]ata[Vv]olume"
rg -l "wait_deleted.*[Dd]ata[Vv]olume"

# Look for timeout constants
echo -e "\n=== Timeout constants and their usage ==="
rg "TIMEOUT_[1234]MINUTE"

# Look for deletion timeouts and failures
echo -e "\n=== Deletion timeout patterns ==="
rg "TimeoutExpired.*delete"
rg "deletion.*failed"
rg "wait_deleted.*timeout"

Length of output: 19106

"""
Wait until DataVolume and the PVC created by it are deleted

Expand Down