Add comprehensive test suite for frontend and backend#5277
Add comprehensive test suite for frontend and backend#5277MarkusNeusinger merged 6 commits intomainfrom
Conversation
- Fix syntax errors in agentic modules (unparenthesized except clauses) - Add tests for insights helpers (_score_bucket, _flatten_tags, _parse_iso, _collect_impl_tags) - Add tests for SEO helpers (_lastmod, _build_sitemap_xml, BOT_HTML_TEMPLATE) - Add tests for plots filter helpers (all 15 helper functions) - Add tests for API schemas (all 8 Pydantic models) - Add tests for config resolve_model (CLI/tier mapping) - Add tests for database models (Spec, Library, Impl) - Add tests for database repositories (CRUD, upsert, queries) - Add tests for sync_to_postgres helpers (_validate_quality_score, _validate_spec_id, _parse_markdown_section) - Add extended analytics tests (all 24+ platform detection patterns) https://claude.ai/code/session_01KhAhJKpEoqCzmWzcALSfW6
Add comprehensive tests for all previously untested components and pages: Components: - SpecTabs.test.tsx: 22 tests (tabs, toggle, copy, quality, tags) - SpecDetailView.test.tsx: 13 tests (zoom, actions, overlay, counter) - SpecOverview.test.tsx: 13 tests (grid, sorting, cards, tooltips) - FilterBar.test.tsx: 5 tests (chips, counter, filter groups) - Layout.test.tsx: 4 tests (provider, context, fetch) - CodeHighlighter.test.tsx: 3 tests (render, language) Pages: - StatsPage.test.tsx: 9 tests (dashboard, stats, loading, error) - CatalogPage.test.tsx: 8 tests (specs, grouping, loading) - SpecPage.test.tsx: 10 tests (overview/detail modes, 404, fetch) - DebugPage.test.tsx: 4 tests (debug data, loading, error) - HomePage.test.tsx: 4 tests (components, POTD, grid) - InteractivePage.test.tsx: 4 tests (iframe, loading, error) Utils/Hooks: - filters-extended.test.ts: 17 tests (getAvailableValues, search) - useFilterState-extended.test.ts: 7 tests (isFiltersEmpty) https://claude.ai/code/session_01KhAhJKpEoqCzmWzcALSfW6
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This PR expands automated test coverage across the backend (API helper functions, schemas, DB models/repositories, automation helpers, config resolution) and the frontend (React pages/components, hooks, and filter utilities). It also fixes Python exception syntax in the agentic workflow modules to be valid on Python 3.
Changes:
- Added new pytest unit tests for config resolution, DB models/repositories, API helper functions/schemas, analytics detection, and sync helpers.
- Added new Vitest/React Testing Library suites for multiple pages/components/hooks and filter utilities.
- Fixed invalid multi-exception
exceptsyntax inagentic/workflows/modules/*.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/core/test_config_resolve.py | Adds unit tests for Settings.resolve_model tier resolution. |
| tests/unit/core/database/test_repositories.py | Adds repository CRUD/upsert tests and updatable-field set checks. |
| tests/unit/core/database/test_models.py | Adds ORM model constant/default/persistence tests. |
| tests/unit/automation/scripts/test_sync_helpers.py | Adds tests for sync_to_postgres helper validators/parsers. |
| tests/unit/api/test_seo_helpers.py | Adds tests for SEO sitemap/lastmod helpers and bot HTML template. |
| tests/unit/api/test_schemas.py | Adds tests for API Pydantic schema defaults/serialization. |
| tests/unit/api/test_plots_helpers.py | Adds tests for plot filtering helper functions in api/routers/plots.py. |
| tests/unit/api/test_insights_helpers.py | Adds tests for insights helper functions (_score_bucket, _flatten_tags, _parse_iso, _collect_impl_tags). |
| tests/unit/api/test_analytics_extended.py | Adds extended tests for user-agent platform detection. |
| app/src/utils/filters-extended.test.ts | Adds extra unit coverage for filter utility functions. |
| app/src/pages/StatsPage.test.tsx | Adds component tests for Stats page loading/success/error rendering. |
| app/src/pages/SpecPage.test.tsx | Adds component tests for Spec page states and routing behavior. |
| app/src/pages/InteractivePage.test.tsx | Adds component tests for interactive iframe loading/error cases. |
| app/src/pages/HomePage.test.tsx | Adds component tests for Home page layout and conditional rendering. |
| app/src/pages/DebugPage.test.tsx | Adds component tests for Debug page loading/success/error states. |
| app/src/pages/CatalogPage.test.tsx | Adds component tests for Catalog page fetch and rendering behavior. |
| app/src/hooks/useFilterState-extended.test.ts | Adds extended tests for isFiltersEmpty. |
| app/src/components/SpecTabs.test.tsx | Adds comprehensive tests for SpecTabs interactions, rendering, and analytics hooks. |
| app/src/components/SpecOverview.test.tsx | Adds tests for overview card rendering/sorting/actions. |
| app/src/components/SpecDetailView.test.tsx | Adds tests for detail view image/actions/interactive link/zoom behavior. |
| app/src/components/Layout.test.tsx | Adds tests for Layout/AppDataProvider rendering and initial fetch behavior. |
| app/src/components/FilterBar.test.tsx | Adds tests for FilterBar rendering and chip display. |
| app/src/components/CodeHighlighter.test.tsx | Adds tests for CodeHighlighter rendering and language setup. |
| agentic/workflows/modules/state.py | Fixes invalid except A, B: syntax to tuple form. |
| agentic/workflows/modules/orchestrator.py | Fixes invalid except A, B: syntax to tuple form. |
| agentic/workflows/modules/agent.py | Fixes invalid except A, B: syntax to tuple form in multiple blocks. |
| from core.database.models import Impl, Library, Spec | ||
| from core.database.repositories import ( | ||
| IMPL_UPDATABLE_FIELDS, | ||
| LIBRARY_UPDATABLE_FIELDS, | ||
| SPEC_UPDATABLE_FIELDS, | ||
| BaseRepository, | ||
| ImplRepository, | ||
| LibraryRepository, | ||
| SpecRepository, | ||
| ) |
There was a problem hiding this comment.
Unused imports Impl and BaseRepository are never referenced in this test file and will fail Ruff (F401). Please remove them from the import list (or use them in assertions if intended).
| import pytest | ||
|
|
There was a problem hiding this comment.
pytest is imported but never used in this module; with Ruff enabled this will fail due to an unused import (F401). Remove the unused import.
| import pytest |
| import pytest | ||
|
|
There was a problem hiding this comment.
pytest is imported but not used anywhere in this test module; Ruff will flag this as an unused import (F401). Please remove it.
| import pytest |
| import pytest | ||
|
|
There was a problem hiding this comment.
pytest is imported but unused in this module; Ruff will raise F401. Remove the unused import to keep lint passing.
| import pytest |
| import pytest | ||
|
|
There was a problem hiding this comment.
pytest is imported but never used in this file; Ruff will fail with F401 (unused import). Please remove it.
| import pytest |
| import pytest | ||
|
|
There was a problem hiding this comment.
pytest is imported but unused in this module; Ruff will flag this as F401. Remove the unused import.
| import pytest |
| import pytest | ||
|
|
There was a problem hiding this comment.
pytest is imported but unused in this test module; Ruff will report an unused import (F401). Please remove it.
| import pytest |
| import { describe, it, expect, vi } from 'vitest'; | ||
| import { | ||
| getCounts, | ||
| getSelectedValuesForCategory, | ||
| getAvailableValues, | ||
| getAvailableValuesForGroup, | ||
| getSearchResults, | ||
| } from './filters'; |
There was a problem hiding this comment.
Unused imports (vi, getCounts, and getSelectedValuesForCategory) will trigger the frontend ESLint rule @typescript-eslint/no-unused-vars. Please remove the unused imports or add coverage that uses them.
| // Mock window.location.href setter | ||
| const hrefSetter = vi.fn(); | ||
| Object.defineProperty(window, 'location', { | ||
| value: { href: '' }, | ||
| writable: true, | ||
| }); | ||
| Object.defineProperty(window.location, 'href', { | ||
| set: hrefSetter, | ||
| get: () => '', | ||
| }); |
There was a problem hiding this comment.
This test mutates window.location but never restores it. That can leak state into subsequent tests (and can be fragile in jsdom if location isn't configurable). Please capture the original window.location, define it with { configurable: true }, and restore it in an afterEach (similar to other tests in the repo).
| def settings(self) -> Settings: | ||
| """Create a Settings instance with default model mappings.""" | ||
| with patch.dict("os.environ", {}, clear=False): | ||
| return Settings( |
There was a problem hiding this comment.
patch.dict("os.environ", {}, clear=False) doesn't actually isolate the test from environment variables (it leaves existing env vars in place). Since Settings reads from os.environ, these tests can become non-deterministic if CI/dev shells set CLI_MODEL_* vars. Consider using clear=True (or explicitly unsetting the relevant keys) in these contexts.
- Remove unused pytest imports from 6 Python test files - Remove unused Impl/BaseRepository imports from test_repositories.py - Remove unused vi/getCounts/getSelectedValuesForCategory from filters-extended.test.ts - Fix global -> globalThis in SpecTabs.test.tsx for ESLint compatibility - Add configurable: true to window.location mock in SpecTabs.test.tsx - Use clear=True in patch.dict for test isolation in test_config_resolve.py - Use precise <loc> assertions in test_seo_helpers.py to fix CodeQL URL warnings - Fix import sorting via ruff auto-fix - Remove unused datetime import in test_insights_helpers.py https://claude.ai/code/session_01KhAhJKpEoqCzmWzcALSfW6
- Auto-format 7 files with ruff format (test files + agentic modules) - Fix import sorting in test_models.py (ruff I001) https://claude.ai/code/session_01KhAhJKpEoqCzmWzcALSfW6
| """ | ||
| Tests for Settings.resolve_model method. | ||
|
|
||
| Tests model tier resolution for different CLIs. | ||
| """ |
There was a problem hiding this comment.
PR description mentions fixing Python 3 syntax errors in agentic/workflows/modules/{orchestrator,state,agent}.py, but those files still contain invalid exception syntax (e.g., except json.JSONDecodeError, ValueError:). Either include those fixes in this PR or update the PR description so it matches the actual changes; as-is, the repo still has unimportable Python modules.
| from automation.scripts.sync_to_postgres import _parse_markdown_section, _validate_quality_score, _validate_spec_id | ||
|
|
||
|
|
There was a problem hiding this comment.
This test imports private helpers directly from automation.scripts.sync_to_postgres, which executes side effects at import time (e.g., load_dotenv() and sys.path manipulation). That makes the unit test environment-dependent and can leak state to other tests; consider moving these pure helpers into a side-effect-free module (and importing from there) or gating the side effects behind a main()/if __name__ == '__main__' block.
| from automation.scripts.sync_to_postgres import _parse_markdown_section, _validate_quality_score, _validate_spec_id | |
| import ast | |
| from pathlib import Path | |
| def _load_sync_helpers() -> tuple[object, object, object]: | |
| """Load helper functions from sync_to_postgres without importing the script module.""" | |
| module_path = Path(__file__).resolve().parents[4] / "automation" / "scripts" / "sync_to_postgres.py" | |
| source = module_path.read_text(encoding="utf-8") | |
| tree = ast.parse(source, filename=str(module_path)) | |
| helper_names = { | |
| "_parse_markdown_section", | |
| "_validate_quality_score", | |
| "_validate_spec_id", | |
| } | |
| selected_nodes = [ | |
| node | |
| for node in tree.body | |
| if isinstance(node, ast.FunctionDef) and node.name in helper_names | |
| ] | |
| module = ast.Module(body=selected_nodes, type_ignores=[]) | |
| namespace = {"__builtins__": __builtins__, "__name__": "_sync_to_postgres_helpers_for_tests"} | |
| exec(compile(module, filename=str(module_path), mode="exec"), namespace) | |
| return ( | |
| namespace["_parse_markdown_section"], | |
| namespace["_validate_quality_score"], | |
| namespace["_validate_spec_id"], | |
| ) | |
| _parse_markdown_section, _validate_quality_score, _validate_spec_id = _load_sync_helpers() |
| // Mock window.location.href setter to prevent jsdom navigation errors. | ||
| // Use configurable: true so the property can be redefined if needed. | ||
| const hrefSetter = vi.fn(); | ||
| Object.defineProperty(window, 'location', { | ||
| value: { href: '' }, | ||
| writable: true, | ||
| configurable: true, | ||
| }); | ||
| Object.defineProperty(window.location, 'href', { | ||
| set: hrefSetter, | ||
| get: () => '', | ||
| configurable: true, | ||
| }); | ||
|
|
||
| render( | ||
| <SpecTabs | ||
| {...baseProps} | ||
| onTrackEvent={onTrackEvent} | ||
| tags={{ plot_type: ['scatter'] }} | ||
| />, | ||
| ); | ||
|
|
||
| await user.click(screen.getByText('scatter')); | ||
|
|
||
| expect(onTrackEvent).toHaveBeenCalledWith('tag_click', { | ||
| param: 'plot', | ||
| value: 'scatter', | ||
| source: 'spec_detail', | ||
| }); |
There was a problem hiding this comment.
This test permanently redefines window.location/window.location.href without restoring the original object. That can leak global state and make other tests flaky (especially if the test runner doesn't fully isolate environments); capture the original window.location (or the href descriptor) and restore it in an afterEach/finally block.
| // Mock window.location.href setter to prevent jsdom navigation errors. | |
| // Use configurable: true so the property can be redefined if needed. | |
| const hrefSetter = vi.fn(); | |
| Object.defineProperty(window, 'location', { | |
| value: { href: '' }, | |
| writable: true, | |
| configurable: true, | |
| }); | |
| Object.defineProperty(window.location, 'href', { | |
| set: hrefSetter, | |
| get: () => '', | |
| configurable: true, | |
| }); | |
| render( | |
| <SpecTabs | |
| {...baseProps} | |
| onTrackEvent={onTrackEvent} | |
| tags={{ plot_type: ['scatter'] }} | |
| />, | |
| ); | |
| await user.click(screen.getByText('scatter')); | |
| expect(onTrackEvent).toHaveBeenCalledWith('tag_click', { | |
| param: 'plot', | |
| value: 'scatter', | |
| source: 'spec_detail', | |
| }); | |
| const originalLocation = window.location; | |
| try { | |
| // Mock window.location.href setter to prevent jsdom navigation errors. | |
| // Use configurable: true so the property can be redefined if needed. | |
| const hrefSetter = vi.fn(); | |
| Object.defineProperty(window, 'location', { | |
| value: { href: '' }, | |
| writable: true, | |
| configurable: true, | |
| }); | |
| Object.defineProperty(window.location, 'href', { | |
| set: hrefSetter, | |
| get: () => '', | |
| configurable: true, | |
| }); | |
| render( | |
| <SpecTabs | |
| {...baseProps} | |
| onTrackEvent={onTrackEvent} | |
| tags={{ plot_type: ['scatter'] }} | |
| />, | |
| ); | |
| await user.click(screen.getByText('scatter')); | |
| expect(onTrackEvent).toHaveBeenCalledWith('tag_click', { | |
| param: 'plot', | |
| value: 'scatter', | |
| source: 'spec_detail', | |
| }); | |
| } finally { | |
| Object.defineProperty(window, 'location', { | |
| value: originalLocation, | |
| writable: true, | |
| configurable: true, | |
| }); | |
| } |
| // avg_quality_score: 82.5 => formatNum(82.5) => "82.5" (toLocaleString) | ||
| expect(screen.getByText('82.5')).toBeInTheDocument(); | ||
| // coverage_percent: 73 => "73%" | ||
| expect(screen.getByText('73%')).toBeInTheDocument(); |
There was a problem hiding this comment.
This assertion is locale-sensitive: StatsPage formats values <1000 using toLocaleString() without a fixed locale, so the rendered decimal separator/grouping can differ across environments. To avoid flaky tests, assert more loosely (e.g., regex/number presence) or update the production formatter to pass an explicit locale (and then assert that).
| // avg_quality_score: 82.5 => formatNum(82.5) => "82.5" (toLocaleString) | |
| expect(screen.getByText('82.5')).toBeInTheDocument(); | |
| // coverage_percent: 73 => "73%" | |
| expect(screen.getByText('73%')).toBeInTheDocument(); | |
| // avg_quality_score: 82.5 => formatNum(82.5) => locale-sensitive decimal separator | |
| expect(screen.getByText(/82[.,]5/)).toBeInTheDocument(); | |
| // coverage_percent: 73 => locale formatting may include spacing before the percent sign | |
| expect(screen.getByText(/73\s*%/)).toBeInTheDocument(); |
- Fix Python 2-style except syntax in orchestrator.py, agent.py, state.py: `except X, Y:` -> `except (X, Y):` with fmt: skip to prevent ruff formatter bug from reverting the fix - Make StatsPage quality score assertion locale-tolerant (82.5 -> /82[.,]5/) https://claude.ai/code/session_01KhAhJKpEoqCzmWzcALSfW6
Summary
This PR adds extensive test coverage across the frontend (React components and utilities) and backend (API routes, database models, and helper functions). The test suite includes unit tests for core functionality, integration tests for API endpoints, and component tests for UI elements.
Key Changes
Frontend Tests (React/TypeScript)
SpecTabs,SpecOverview,SpecDetailView,CodeHighlighter,FilterBar, andLayoutcomponentsHomePage,CatalogPage,SpecPage,StatsPage,InteractivePage, andDebugPageuseFilterStatehookfilters-extended.test.ts)Backend Tests (Python)
API Tests:
test_plots_helpers.py: Tests for plot filtering helper functions (_get_category_values,_category_matches_filter,_image_matches_groups, etc.)test_insights_helpers.py: Tests for insights helper functions (_score_bucket,_flatten_tags,_collect_impl_tags,_parse_iso)test_seo_helpers.py: Tests for SEO helper functions (_lastmod,_build_sitemap_xml)test_schemas.py: Tests for Pydantic API schemas (ImplementationResponse,SpecDetailResponse,FilterCountsResponse, etc.)test_analytics_extended.py: Extended tests for analytics platform detectionDatabase Tests:
test_models.py: Tests for ORM models (Spec,Library,Impl) including model constants and persistencetest_repositories.py: Tests for repository classes (SpecRepository,LibraryRepository,ImplRepository) with in-memory SQLiteAutomation Tests:
test_sync_helpers.py: Tests for sync helper functions (_validate_quality_score,_parse_markdown_section,_validate_spec_id)Config Tests:
test_config_resolve.py: Tests forSettings.resolve_modelmethod for different CLI model tier resolutionBug Fixes
agentic/workflows/modules/orchestrator.py: Changedexcept json.JSONDecodeError, ValueError:toexcept (json.JSONDecodeError, ValueError):agentic/workflows/modules/state.pyagentic/workflows/modules/agent.pyNotable Implementation Details
https://claude.ai/code/session_01KhAhJKpEoqCzmWzcALSfW6