fix: Database partition management commands

[ElioDiNino]

• Added the pgpartition command in delete mode for all development scripts
• Removed the old dev.sh script
• Added the pgpartition command in add-only mode for the worker and enterprise scripts; TBD if these are actually used

[codecov-notifications]

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.34%. Comparing base (b24e2e8) to head (d321951).

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #263   +/-   ##
=======================================
  Coverage   94.34%   94.34%           
=======================================
  Files        1223     1223           
  Lines       45305    45305           
  Branches     1441     1441           
=======================================
  Hits        42744    42744           
  Misses       2260     2260           
  Partials      301      301           

Flag	Coverage Δ
apiunit	96.48% <ø> (ø)
workerintegration	61.63% <ø> (ø)
workerunit	90.61% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[seer-by-sentry]

Sentry detected 4 potential issues in your recent changes

A potential issue exists in enterprise.sh where pgpartition is called directly as a Python script (python pgpartition.py) instead of a Django management command (python manage.py pgpartition). This could lead to a FileNotFoundError and prevent the application from starting.

• Description: When the enterprise.sh script executes python pgpartition.py --yes --skip-delete, the Python interpreter will search for a file named pgpartition.py in the current directory. As no such file exists in the codebase, this will result in a FileNotFoundError. For example, if the script attempts to run python non_existent_script.py, Python will immediately raise this error, halting execution. This issue is triggered directly by the incorrect command syntax, specifically the omission of manage.py, which is necessary to invoke pgpartition as a Django management command. This will cause the startup process to terminate unexpectedly.
• Code location: apps/codecov-api/enterprise.sh:33~44
• Suggested fix: Modify apps/codecov-api/enterprise.sh at lines 33 and 44. Change python pgpartition.py to python manage.py pgpartition. This ensures the pgpartition command is correctly invoked as a Django management command, resolving the FileNotFoundError.

Suspicion of a FileNotFoundError in apps/codecov-api/enterprise.sh at lines 33 and 44. The script attempts to execute a non-existent pgpartition.py file, which could prevent enterprise API servers from starting.

• Description: The enterprise.sh script attempts to execute python pgpartition.py --yes --skip-delete. However, a file named pgpartition.py does not exist anywhere in the codebase. Python will raise a FileNotFoundError when attempting to execute this non-existent script. This failure occurs in critical startup paths (lines 33 and 44) right after migrations and before starting the API server. Since the shell script lacks error handling, the non-zero exit status from the failed Python command will cause the entire enterprise startup sequence to abort, preventing the server from starting properly. For example, if the script is run, the shell will try to execute /usr/bin/python pgpartition.py and fail with an error like bash: pgpartition.py: command not found or python: can't open file 'pgpartition.py': [Errno 2] No such file or directory.
• Code location: apps/codecov-api/enterprise.sh:33~44
• Suggested fix: Change python pgpartition.py to python manage.py pgpartition in apps/codecov-api/enterprise.sh at lines 33 and 44 to correctly invoke the Django management command.

There is a suspicion that the enterprise.sh script at lines 33 and 44 incorrectly calls python pgpartition.py instead of python manage.py pgpartition. This could lead to a FileNotFoundError because pgpartition.py does not exist, potentially causing the script to fail and impacting system startup.

• Description: The enterprise.sh script attempts to execute python pgpartition.py --yes --skip-delete at lines 33 and 44. However, there is no pgpartition.py file in the codebase. The pgpartition command is a Django management command provided by the django-postgres-extra library, and it should be invoked using python manage.py pgpartition. This incorrect invocation will lead to a FileNotFoundError when the shell attempts to execute the non-existent Python file. For example, if the script runs, the system will encounter an error similar to FileNotFoundError: [Errno 2] No such file or directory: 'pgpartition.py'. This will prevent the enterprise startup script from completing successfully, leading to unexpected behavior in the production environment. All other scripts in the codebase correctly use the python manage.py format for this command.
• Code location: apps/codecov-api/enterprise.sh:33~44
• Suggested fix: Modify apps/codecov-api/enterprise.sh at lines 33 and 44. Change python pgpartition.py --yes --skip-delete to python manage.py pgpartition --yes --skip-delete. This ensures the correct Django management command is invoked, resolving the FileNotFoundError.

Potential race condition during concurrent pgpartition command execution in apps/worker/worker.sh:17 and apps/codecov-api/enterprise.sh could lead to database inconsistencies and unexpected startup failures.

• Description: The pgpartition command is executed concurrently by multiple services (API and worker) during startup, specifically at apps/worker/worker.sh:17 and apps/codecov-api/enterprise.sh:33~44. While the command itself is designed to manage partitions, the lack of a distributed coordination mechanism (e.g., advisory locks) between these concurrent executions creates a race condition. Historical data shows PostgresPartitioningError (e.g., 'Missing upload_breadcrumbs_uploadbreadcrumb' table) can occur when pgpartition commands fail due to inconsistent database states, likely from migration timing issues or concurrent operations. If such a failure occurs during startup, and given the absence of explicit error handling in the shell scripts, it could lead to an unexpected startup failure, preventing the service from becoming fully operational. For example, if both API and worker attempt to create the same partition simultaneously, one might fail, leading to an inconsistent state that prevents further startup.
• Code location: apps/worker/worker.sh:17
• Suggested fix: Implement a distributed locking mechanism (e.g., Redis-based lock) around pgpartition command invocations in startup scripts to ensure only one instance performs partition management at a time.

_{Did you find this useful? React with a 👍 or 👎}