-
Notifications
You must be signed in to change notification settings - Fork 0
Fixes for multiple backup runs #28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements comprehensive fixes to prevent multiple backup runs for the same repository, addressing issues with duplicate backup job creation and execution. The changes add multiple layers of protection including database-level checks, file-based locking, and automatic cleanup mechanisms.
Key changes:
- Added file-based locking mechanism to prevent concurrent backup executions
- Implemented automatic cleanup of orphaned temp directories and duplicate jobs
- Enhanced duplicate prevention with extended time windows and stuck job detection
- Added periodic health check job to monitor and clean up scheduler inconsistencies
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
File | Description |
---|---|
backup_service.py | Added auto-cleanup for orphaned temp directories before starting new backups |
app.py | Extensive changes including file-based locking, job tracking, automatic cleanup of duplicates and stuck jobs, and periodic health monitoring |
Comments suppressed due to low confidence (1)
app.py:1
- The code references
BackupJob.created_at
field, but the BackupJob model may not have this field based on the context. The visible fields in the backup job creation arestarted_at
,status
, etc. This could cause an AttributeError.
import os
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
# Auto-cleanup: Check for and clean up any orphaned temp directories | ||
user_backup_dir = self.backup_base_dir / f"user_{repository.user_id}" | ||
repo_backup_dir = user_backup_dir / repository.name | ||
if repo_backup_dir.exists(): | ||
self._cleanup_temp_directories(repo_backup_dir) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The method _cleanup_temp_directories
is called but not defined in the shown code. This will result in an AttributeError when the backup service attempts to clean up temp directories.
Copilot uses AI. Check for mistakes.
try: | ||
os.unlink(lock_file_path) | ||
except: | ||
pass |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using bare except:
clause suppresses all exceptions including system exits and keyboard interrupts. Use a specific exception type like OSError
or FileNotFoundError
instead.
try: | |
os.unlink(lock_file_path) | |
except: | |
pass | |
try: | |
os.unlink(lock_file_path) | |
except OSError: | |
pass |
Copilot uses AI. Check for mistakes.
try: | ||
lock_file = open(lock_file_path, 'w') | ||
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) | ||
logger.info(f"Acquired file lock for repository {repo.name}") | ||
|
||
try: | ||
logger.info(f"Starting scheduled backup for repository: {repo.name}") | ||
backup_service.backup_repository(repo) | ||
logger.info(f"Completed scheduled backup for repository: {repo.name}") | ||
finally: | ||
fcntl.flock(lock_file.fileno(), fcntl.LOCK_UN) | ||
lock_file.close() | ||
try: | ||
os.unlink(lock_file_path) | ||
except: | ||
pass | ||
|
||
except (IOError, OSError) as lock_error: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The file handle should be managed with a context manager (with
statement) to ensure proper cleanup even if an exception occurs before the explicit close() call.
try: | |
lock_file = open(lock_file_path, 'w') | |
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) | |
logger.info(f"Acquired file lock for repository {repo.name}") | |
try: | |
logger.info(f"Starting scheduled backup for repository: {repo.name}") | |
backup_service.backup_repository(repo) | |
logger.info(f"Completed scheduled backup for repository: {repo.name}") | |
finally: | |
fcntl.flock(lock_file.fileno(), fcntl.LOCK_UN) | |
lock_file.close() | |
try: | |
os.unlink(lock_file_path) | |
except: | |
pass | |
except (IOError, OSError) as lock_error: | |
try: | |
with open(lock_file_path, 'w') as lock_file: | |
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) | |
logger.info(f"Acquired file lock for repository {repo.name}") | |
try: | |
logger.info(f"Starting scheduled backup for repository: {repo.name}") | |
backup_service.backup_repository(repo) | |
logger.info(f"Completed scheduled backup for repository: {repo.name}") | |
finally: | |
fcntl.flock(lock_file.fileno(), fcntl.LOCK_UN) | |
try: | |
os.unlink(lock_file_path) | |
except: | |
pass | |
except (IOError, OSError) as lock_error: |
Copilot uses AI. Check for mistakes.
# Additional check: ensure no backup started in the last 30 seconds to prevent rapid duplicates | ||
recent_cutoff = datetime.utcnow() - timedelta(seconds=30) | ||
# 2. Check for very recent backups (within last 2 minutes) to prevent rapid duplicates | ||
recent_cutoff = datetime.utcnow() - timedelta(minutes=2) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] The time window for preventing duplicates has changed from 30 seconds to 2 minutes without explanation. This magic number should be made configurable or at least documented why 2 minutes was chosen.
Copilot uses AI. Check for mistakes.
No description provided.