Development

[ooemperor]

This pull request introduces several improvements and refactorings across workflow configuration, SQL query generation, and pipeline execution. The main highlights include parallelization of pipeline runs, enhanced SQL query logic for better correctness, updates to related tests, and renaming of GitHub Actions job names for clarity.

Pipeline Execution Improvements:

• Added parallel execution for pipelines in both InbPackage and SystemPackage by using Go's sync.WaitGroup, allowing all table pipelines to run concurrently and improving performance. (pkg/packages/inbrdv/InbPackage.go [1] pkg/packages/srcinb/SystemPackage.go [2]

SQL Query Logic Enhancements:

• Refactored the delete query in BuildInbRdvSatDeleteQuery to use a NOT EXISTS clause and table aliases, ensuring more accurate deletion logic. (pkg/builder/InbRdv.go pkg/builder/InbRdv.goL56-R63)
• Improved the insert query in BuildInbRdvSatInsertQuery to use a LEFT JOIN and select only new records, increasing correctness and efficiency. (pkg/builder/InbRdv.go pkg/builder/InbRdv.goL74-R80)

Test Updates:

• Updated unit tests to match the new SQL query formats for both delete and insert operations, ensuring test coverage reflects the improved logic. (pkg/builder/InbRdv_test.go [1] [2] pkg/pipeline/inbrdv/RdvPipelineBuilder_test.go [3] [4]
• Adjusted the expected output in ScriptTransactionWrapper and its test to include an explicit COMMIT; statement, ensuring transaction completion. (pkg/builder/Global.go [1] pkg/builder/Global_test.go [2]

Workflow Configuration:

• Renamed GitHub Actions job names for better clarity and consistency, e.g., build to build-test, build-publish-development, build-publish-main, and go-test. (.github/workflows/docker_build.yml [1] .github/workflows/docker_build_publish_development.yml [2] .github/workflows/docker_build_publish_main.yml [3] .github/workflows/go_test.yml [4]

Minor Codebase Maintenance:

• Added missing imports for the sync package where parallelization was implemented. (pkg/packages/inbrdv/InbPackage.go [1] pkg/packages/srcinb/SystemPackage.go [2]