chore: Dev to main

[NirajC-Microsoft]

Purpose

This pull request introduces a comprehensive suite of unit tests for the core data models and pipeline entities in the ContentProcessor backend, significantly improving test coverage and reliability. It also refines the coverage configuration and test workflow to ensure more accurate metrics. The most important changes are grouped below:

Testing: New Unit Tests for Core Modules

• Added thorough unit tests for the following modules: pipeline_file, pipeline_step_result, pipeline_data, schema, and content_process. These tests cover model creation, method behaviors, error handling, and interactions with external dependencies (e.g., storage and database helpers). [1] [2] [3] [4] [5]

Testing: Enhancements to Existing Tests

• Expanded tests in test_pipeline_data.py to cover the DataPipeline class, including object parsing, file management, step result retrieval, and persistent storage interactions.
• Added imports for DataPipeline and ArtifactType to support new and enhanced tests in test_pipeline_data.py.

Testing: Coverage and Workflow Improvements

• Updated the pytest coverage configuration in .github/workflows/test.yml to use the correct source directory (src) for coverage reporting, ensuring accurate measurement.
• Added a [tool.coverage.run] and [tool.coverage.report] section to pyproject.toml to specify coverage source, omit test files, and exclude certain lines from coverage, further refining the coverage metrics.

Testing: Clean-up and Organization

• Removed obsolete and commented-out test code from test_pipeline_data.py to maintain clarity and focus on active tests.

These changes collectively strengthen the backend's reliability by ensuring that core entities and their interactions are robustly tested and that coverage metrics are precise and meaningful.

Does this introduce a breaking change?

• Yes
• No

Golden Path Validation

• I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• I have validated the deployment process successfully and all services are running as expected with this change.

What to Check

Verify that the following are valid

• ...

Other Information