chore: Main merge to Demo

[Shreyas-Microsoft]

Purpose

This pull request introduces significant improvements to CI/CD workflows, infrastructure automation, and documentation. The main changes are the addition of a new test workflow for the processor service, enhancements to parameter validation and notification for Bicep deployments, improved tagging and resource management in Bicep templates, and minor documentation and dependency updates.

CI/CD Workflow Enhancements:

• Added a new processor_tests job to .github/workflows/test.yml to run tests and coverage for the src/processor service, mirroring the backend API test process. This includes coverage threshold enforcement, test file detection, and automated coverage reporting. [1] [2] [3]
• Updated the backend API test job to install pytest-asyncio, enforce a minimum coverage threshold, and prefix coverage XML filenames for better reporting. [1] [2] [3]

Infrastructure Automation and Bicep Improvements:

• Enhanced the Bicep parameter validation workflow to generate and use an HTML email body for notifications, include additional environment variables, and attach new artifacts for improved notification formatting and clarity. [1] [2] [3]
• Updated Bicep modules and templates (infra/main.bicep, infra/main_custom.bicep, infra/main.json, etc.) to propagate tags more comprehensively to resources, update role definition comments, and ensure correct dependency ordering in generated ARM templates. [1] [2] [3] [4] [5] [6] [7]
• Updated Bicep module generator metadata to reflect the latest Bicep CLI version. [1] [2] [3] [4]

Other Notable Changes:

• Added Bicep CLI to the list of required tools in the deployment documentation.
• Updated role definition comments from "Azure AI User" to "Foundry User" for clarity. [1] [2] [3]
• Removed the unused enableAnalyticalStorage parameter from the Cosmos DB Bicep module.
• Clarified a comment in the Python parameter parsing script for better maintainability.

Does this introduce a breaking change?

• Yes
• No

Golden Path Validation

• I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• I have validated the deployment process successfully and all services are running as expected with this change.

What to Check

Verify that the following are valid

• ...

Other Information

[Copilot]

Pull request overview

This PR merges mainline changes into the demo branch, focusing on CI coverage expansion (including src/processor), improved deployment/infra parameter validation + notifications, broader tag propagation in Bicep templates, and a large set of new unit tests across the processor and backend API.

Changes:

• Extended GitHub Actions testing to run processor unit tests with coverage reporting and added/adjusted coverage threshold enforcement.
• Improved infra automation: Bicep parameter validation now emits an HTML email body; Bicep templates propagate tags more consistently; ARM template regeneration updated.
• Added substantial unit test coverage for processor and backend-api modules; updated a few runtime components to log best-effort exception paths instead of silently swallowing.

Reviewed changes

Copilot reviewed 104 out of 115 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
.github/workflows/test.yml	Adds processor test+coverage job and adjusts backend job coverage behavior/reporting.
.github/workflows/validate-bicep-params.yml	Enhances scheduled param validation to generate HTML email artifacts and send formatted notifications.
docs/DeploymentGuide.md	Documents Bicep CLI as a required tool.
infra/main.bicep	Propagates tags to container apps environment and updates role comment text.
infra/main_custom.bicep	Mirrors tag propagation and role comment updates for the custom template.
infra/main.parameters.json	Adds tags parameter wiring for deployments.
infra/main.waf.parameters.json	Adds tags parameter wiring for WAF deployments.
infra/main.json	Regenerated ARM template (updated generator metadata, dependsOn ordering, and tags wiring).
infra/modules/cosmosDb.bicep	Removes unused enableAnalyticalStorage param passthrough.
src/backend-api/pyproject.toml	Pins/updates dependencies, adds pytest-asyncio, and coverage omit config.
src/backend-api/src/app/application.py	DI registration tweak (uses class directly instead of lambda factory).
src/backend-api/src/app/libs/base/SKLogicBase.py	Type improvements (Any) and overload bodies changed to pass.
src/backend-api/src/app/libs/base/fastapi_protocol.py	Protocol stub method body changed to pass.
src/backend-api/src/tests/application/test_application.py	Adds bootstrap coverage for backend Application wiring.
src/backend-api/src/tests/application/test_application_context_extra.py	Adds more AppContext DI lifecycle tests.
src/backend-api/src/tests/application/test_dependency_injection.py	Updates singleton registration test to match new DI registration pattern.
src/backend-api/src/tests/azure/test_app_configuration_helper.py	Adds tests for App Configuration helper behavior.
src/backend-api/src/tests/base/test_kernel_agent.py	Adds isolated tests for kernel_agent with stubs/shims.
src/backend-api/src/tests/base/test_sk_logic_base.py	Adds tests for SKLogicBase import quirks and abstract contract branches.
src/backend-api/src/tests/base/test_typed_fastapi.py	Adds coverage for TypedFastAPI and protocol helper utilities.
src/backend-api/src/tests/model/test_entities.py	Adds model/entity default-value tests.
src/backend-api/src/tests/model/test_router_models.py	Adds router model validation/serialization tests.
src/backend-api/src/tests/repositories/test_process_status_repository.py	Adds tests for status rendering helpers and repository snapshot behavior.
src/backend-api/src/tests/repositories/test_repositories_extra.py	Adds tests ensuring repositories set updated_at and call super.
src/backend-api/src/tests/routers/test_http_probes.py	Adds tests for health/startup probe routes.
src/backend-api/src/tests/routers/test_router_debug.py	Adds tests for debug config route payload.
src/backend-api/src/tests/routers/test_router_files.py	Adds tests for upload route behavior, including filename sanitization.
src/backend-api/src/tests/sas/storage/blob/test_config.py	Adds tests for blob storage helper configuration behavior.
src/backend-api/src/tests/sas/storage/test_shared_config.py	Adds tests for shared storage config behavior.
src/backend-api/src/tests/services/test_auth.py	Adds tests for header-based auth/user extraction.
src/backend-api/src/tests/services/test_implementations.py	Adds tests for concrete service implementations.
src/backend-api/src/tests/services/test_input_validation.py	Adds tests for UUID validation helper.
src/backend-api/src/tests/services/test_interfaces.py	Adds coverage-only tests exercising ABC no-op bodies.
src/backend-api/src/tests/services/test_process_services.py	Adds tests for blob/queue process service behavior.
src/backend-api/src/tests/test_app_init.py	Adds trivial coverage for app package import bootstrap.
src/backend-api/src/tests/test_main.py	Adds tests for main.get_app() singleton caching.
src/frontend/package.json	Updates dependencies and adds overrides.
src/frontend/requirements.txt	Bumps python-multipart version.
src/frontend/frontend_server.py	Removes unused imports/vars in config endpoint.
src/frontend/vite.config.js	Removes unused path import.
src/frontend/src/main.jsx	Adjusts config fetch/fallback flow initialization.
src/frontend/src/api/utils.tsx	Fixes error-file filtering logic to check > 0.
src/frontend/src/commonComponents/ProgressModal/progressModal.tsx	Progress calc ordering tweak.
src/frontend/src/components/batchHistoryPanel.tsx	Formatting cleanup and removes unused categorization function.
src/frontend/src/components/bottomBar.tsx	Removes unused React state import.
src/frontend/src/components/uploadButton.tsx	Removes unused imports and dead handler/state.
src/frontend/src/msal-auth/apiHeaders.ts	Removes unused import.
src/frontend/src/pages/batchView.tsx	Removes unused state and changes summary rendering conditions.
src/frontend/src/pages/landingPage.tsx	Removes unused Fluent imports and unused callback code.
src/frontend/src/pages/modernizationPage.tsx	Cleans unused code/state; fixes file selection matching semantics.
src/frontend/src/pages/processPage.tsx	Removes unused imports/state.
src/processor/pyproject.toml	Dependency bumps, adds coverage omit config, and uv overrides.
src/processor/package.json	Adds package metadata and bumps mermaid; adds dompurify/lodash-es.
src/processor/src/libs/agent_framework/agent_framework_helper.py	Changes overload stubs from ... to pass (coverage-related).
src/processor/src/libs/agent_framework/azure_openai_response_retry.py	Retries additional transient error signal and logs best-effort stream close failures.
src/processor/src/libs/agent_framework/qdrant_memory_store.py	Logs best-effort delete failures instead of silent pass.
src/processor/src/libs/base/orchestrator_base.py	Increases ResultGenerator max_tokens and adds rationale comment.
src/processor/src/libs/mcp_server/datetime/mcp_datetime.py	Removes unreachable returns in error branches.
src/processor/src/libs/mcp_server/mermaid/mcp_mermaid.py	Clarifies JSON decode failure comment.
src/processor/src/libs/reporting/migration_report_generator.py	Logs file size retrieval failures instead of silently ignoring.
src/processor/src/services/control_api.py	Logs best-effort stop/cleanup failures.
src/processor/src/services/queue_service.py	Replaces silent passes with debug logging in best-effort cleanup paths.
src/processor/src/steps/migration_processor.py	Adds debug logs for summary generation/telemetry failures; ensures explicit return None fallback.
src/processor/src/utils/agent_telemetry.py	Removes dead commented blocks; adds explicit None return; adds debug logs for swallowed exceptions.
src/processor/src/tests/conftest.py	Clarifies intentional side-effect import and suppresses unused import warning.
src/processor/src/tests/unit/steps/analysis/test_analysis_executor.py	Makes monkeypatching of text2art resilient to missing import.
src/processor/src/tests/unit/libs/agent_framework/test_azure_openai_response_retry_utils.py	Updates tests to reflect retry/context-length behavior changes.
src/processor/src/tests/unit/** (multiple new files)	Adds broad unit test coverage across utils, orchestrators, DI/context, reporting, and agent framework helpers.

Files not reviewed (2)

• src/frontend/package-lock.json: Language not supported
• src/processor/package-lock.json: Language not supported

Comments suppressed due to low confidence (1)

src/frontend/src/pages/batchView.tsx:573

• batchSummary is typed as BatchSummary | null, but this branch now runs without checking that batchSummary is non-null and immediately accesses batchSummary.error_count. This can throw during initial render/loading. Restore the null guard (e.g., if (selectedFile.id === "summary" && batchSummary)) or use optional chaining/defaults before reading fields.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.