Integrate Automated Tests for Pipeline Dependencies

[eriwarr]

To improve our ability to catch issues early when pipeline dependencies are updated, we need to implement an automated testing process that redeploys services and triggers a test DAG to confirm connections and basic functionality.

Action Items:

• Develop a special test DAG for connection and functionality verification.
• Write a script to redeploy services, trigger the test DAG, and monitor execution within the GitHub Actions.
• Implement error handling and notifications for test DAG failures.

Note:

• The test DAG should be designed to run quickly and provide clear pass/fail results to minimize downtime in the CI/CD pipeline.