From 0d039b7d2d54dc07fcf794a6eb7d6a29dc78a1b7 Mon Sep 17 00:00:00 2001 From: muhab Date: Mon, 15 Sep 2025 14:03:27 +0200 Subject: [PATCH 1/2] doc(req): update every software item spec md file section of itemFulfills [skip:ci] --- specifications/SPEC-APPLICATION-SERVICE.md | 12 ++++----- .../SPEC-BUILD-CHAIN-CICD-SERVICE.md | 3 +-- specifications/SPEC-DATASET-SERVICE.md | 15 ++++++----- .../SPEC-MODULE-SERVICE-TEMPLATE.md | 25 ------------------- specifications/SPEC-UTILS-SERVICE.md | 12 ++++----- specifications/SPEC_GUI_SERVICE.md | 12 ++++----- specifications/SPEC_NOTEBOOK_SERVICE.md | 12 ++++----- specifications/SPEC_PLATFORM_SERVICE.md | 10 ++++---- specifications/SPEC_SYSTEM_SERVICE.md | 12 ++++----- specifications/SPEC_WSI_SERVICE.md | 14 +++++------ 10 files changed, 52 insertions(+), 75 deletions(-) diff --git a/specifications/SPEC-APPLICATION-SERVICE.md b/specifications/SPEC-APPLICATION-SERVICE.md index a467f10c..453f6845 100644 --- a/specifications/SPEC-APPLICATION-SERVICE.md +++ b/specifications/SPEC-APPLICATION-SERVICE.md @@ -1,11 +1,11 @@ --- -itemId: SPEC-APPLICATION-SERVICE +itemId: SPEC-APPLICATION-SERVICE itemTitle: Application Module Specification -itemType: Software Item Spec -itemFulfills: SWR-APPLICATION-1-1, SWR-APPLICATION-1-2 -Module: Application -Layer: Domain Service -Version: 0.2.106 +itemType: Software Item Spec +itemFulfills: SWR-APPLICATION-1-1, SWR-APPLICATION-1-2, SHR-APPLICATION-3, SWR-APPLICATION-2-12 +Module: Application +Layer: Domain Service +Version: 0.2.106 Date: 2025-09-09 --- diff --git a/specifications/SPEC-BUILD-CHAIN-CICD-SERVICE.md b/specifications/SPEC-BUILD-CHAIN-CICD-SERVICE.md index e9d0fe5b..b90dea43 100644 --- a/specifications/SPEC-BUILD-CHAIN-CICD-SERVICE.md +++ b/specifications/SPEC-BUILD-CHAIN-CICD-SERVICE.md @@ -2,8 +2,7 @@ itemId: SPEC-BUILD-CHAIN-CICD-SERVICE itemTitle: Build Chain and CI/CD Module Specification itemType: Software Item Spec -itemFulfills: SWR-BUILD-CHAIN-1 _(Comprehensive Build Chain and CI/CD Pipeline)_ -Module: Build Chain and CI/CD +itemFulfills: TBD _(No infrastructure requirements currently defined)_Module: Build Chain and CI/CD Layer: Infrastructure Service Version: 0.2.140 Date: 2025-09-11 diff --git a/specifications/SPEC-DATASET-SERVICE.md b/specifications/SPEC-DATASET-SERVICE.md index 101d72b3..3ad71761 100644 --- a/specifications/SPEC-DATASET-SERVICE.md +++ b/specifications/SPEC-DATASET-SERVICE.md @@ -1,11 +1,11 @@ --- itemId: SPEC-DATASET-SERVICE itemTitle: Dataset Module Specification -itemType: Software Item Spec -itemFulfills: FE-6386 -Module: Dataset -Layer: Domain Service -Version: 0.2.105 +itemType: Software Item Spec +itemFulfills: SHR-DATASET-1, SWR-DATASET-1-1, SWR-DATASET-1-3 +Module: Dataset +Layer: Domain Service +Version: 0.2.105 Date: 2025-09-11 --- @@ -380,4 +380,7 @@ _Note: For exact version requirements, refer to `pyproject.toml` and dependency **API Documentation**: Auto-generated from docstrings in service classes --- -```` + +``` + +``` diff --git a/specifications/SPEC-MODULE-SERVICE-TEMPLATE.md b/specifications/SPEC-MODULE-SERVICE-TEMPLATE.md index 7868f3ab..90e66c03 100644 --- a/specifications/SPEC-MODULE-SERVICE-TEMPLATE.md +++ b/specifications/SPEC-MODULE-SERVICE-TEMPLATE.md @@ -11,31 +11,6 @@ Date: [DATE] ## Documentation Guidelines -### Code in Specifications - Best Practices - -**INCLUDE Code When:** - -- ✅ Public API signatures (stable contracts) -- ✅ Data structure schemas for inputs/outputs -- ✅ Configuration parameter definitions -- ✅ Error type hierarchies - -**AVOID Code When:** - -- ❌ Internal implementation details -- ❌ Private methods or functions -- ❌ Complete code blocks or algorithms -- ❌ Version-specific dependency details - -**Preferred Approaches:** - -- 📋 Reference interfaces by name and purpose -- 📋 Use schemas (JSON Schema, OpenAPI) for data structures -- 📋 Link to auto-generated documentation for details -- 📋 Focus on behavior and contracts, not implementation - ---- - ## 1. Description ### 1.1 Purpose diff --git a/specifications/SPEC-UTILS-SERVICE.md b/specifications/SPEC-UTILS-SERVICE.md index 120a4da8..ef46032f 100644 --- a/specifications/SPEC-UTILS-SERVICE.md +++ b/specifications/SPEC-UTILS-SERVICE.md @@ -1,11 +1,11 @@ --- -itemId: SPEC-UTILS-SERVICE +itemId: SPEC-UTILS-SERVICE itemTitle: Utils Module Specification -itemType: Software Item Spec -itemFulfills: TBD -Module: Utils -Layer: Infrastructure Service -Version: 0.2.105 +itemType: Software Item Spec +itemFulfills: TBD _(Core infrastructure services - enables other modules to fulfill user requirements)_ +Module: Utils _(Core Infrastructure Services)_ +Layer: Infrastructure Service +Version: 1.0.0 Date: 2025-09-11 --- diff --git a/specifications/SPEC_GUI_SERVICE.md b/specifications/SPEC_GUI_SERVICE.md index 417c7b10..cf257d5a 100644 --- a/specifications/SPEC_GUI_SERVICE.md +++ b/specifications/SPEC_GUI_SERVICE.md @@ -1,11 +1,11 @@ --- -itemId: SPEC-GUI-SERVICE +itemId: SPEC-GUI-SERVICE itemTitle: GUI Module Specification -itemType: Software Item Spec -itemFulfills: TBD -Module: GUI -Layer: Presentation Interface -Version: 0.2.105 +itemType: Software Item Spec +itemFulfills: SHR-APPLICATION-1 +Module: GUI _(Graphical User Interface Framework)_ +Layer: Presentation Interface +Version: 1.0.0 Date: 2025-09-11 --- diff --git a/specifications/SPEC_NOTEBOOK_SERVICE.md b/specifications/SPEC_NOTEBOOK_SERVICE.md index 5b032900..0379fbf4 100644 --- a/specifications/SPEC_NOTEBOOK_SERVICE.md +++ b/specifications/SPEC_NOTEBOOK_SERVICE.md @@ -1,11 +1,11 @@ --- -itemId: SPEC-NOTEBOOK-SERVICE +itemId: SPEC-NOTEBOOK-SERVICE itemTitle: Notebook Module Specification -itemType: Software Item Spec -itemFulfills: TBD -Module: Notebook _(Interactive Data Analysis)_ -Layer: Presentation Interface -Version: 1.0.0 +itemType: Software Item Spec +itemFulfills: SHR-NOTEBOOK-1, SWR-NOTEBOOK-1-1 +Module: Notebook _(Interactive Data Analysis)_ +Layer: Presentation Interface +Version: 1.0.0 Date: 2025-09-11 --- diff --git a/specifications/SPEC_PLATFORM_SERVICE.md b/specifications/SPEC_PLATFORM_SERVICE.md index d13527d9..4933a680 100644 --- a/specifications/SPEC_PLATFORM_SERVICE.md +++ b/specifications/SPEC_PLATFORM_SERVICE.md @@ -1,11 +1,11 @@ --- itemId: SPEC-PLATFORM-SERVICE itemTitle: Platform Module Specification -itemType: Software Item Spec -itemFulfills: TBD - Platform infrastructure requirements (authentication, API client management, health monitoring) -Module: Platform -Layer: Platform Service -Version: 1.0.0 +itemType: Software Item Spec +itemFulfills: SWR-APPLICATION-1-1, SWR-APPLICATION-1-2, SWR-APPLICATION-2-1, SWR-APPLICATION-2-5, SWR-APPLICATION-2-6, SWR-APPLICATION-2-7, SWR-APPLICATION-2-9, SWR-APPLICATION-2-14, SWR-APPLICATION-2-15, SWR-APPLICATION-2-16, SWR-APPLICATION-3-1, SWR-APPLICATION-3-2, SWR-APPLICATION-3-3 +Module: Platform +Layer: Platform Service +Version: 1.0.0 Date: 2025-09-09 --- diff --git a/specifications/SPEC_SYSTEM_SERVICE.md b/specifications/SPEC_SYSTEM_SERVICE.md index 81689119..7aa391fd 100644 --- a/specifications/SPEC_SYSTEM_SERVICE.md +++ b/specifications/SPEC_SYSTEM_SERVICE.md @@ -1,11 +1,11 @@ --- -itemId: SPEC-SYSTEM-SERVICE +itemId: SPEC-SYSTEM-SERVICE itemTitle: System Module Specification -itemType: Software Item Spec -itemFulfills: TBD -Module: System _(Core Platform Services)_ -Layer: Platform Service -Version: 1.0.0 +itemType: Software Item Spec +itemFulfills: SHR-SYSTEM-1, SHR-SYSTEM-2, SHR-SYSTEM-3 +Module: System _(Core Platform Services)_ +Layer: Platform Service +Version: 1.0.0 Date: 2025-09-11 --- diff --git a/specifications/SPEC_WSI_SERVICE.md b/specifications/SPEC_WSI_SERVICE.md index f2586208..fe9e9e15 100644 --- a/specifications/SPEC_WSI_SERVICE.md +++ b/specifications/SPEC_WSI_SERVICE.md @@ -1,12 +1,12 @@ --- -itemId: SPEC-WSI-SERVICE +itemId: SPEC-WSI-SERVICE itemTitle: WSI Module Specification -itemType: Software Item Spec -itemFulfills: TBD -Module: WSI -Layer: Domain Service -Version: 0.2.105 -Date: 2025-09-10 +itemType: Software Item Spec +itemFulfills: SWR-VISUALIZATION-1 +Module: WSI _(Whole Slide Image Processing)_ +Layer: Domain Service +Version: 1.0.0 +Date: 2025-09-11 --- ## 1. Description From 9b082803716197a168ac65259dd4483ba05bbd5e Mon Sep 17 00:00:00 2001 From: muhab Date: Mon, 15 Sep 2025 15:28:30 +0200 Subject: [PATCH 2/2] doc(req): update software item specs to follow template recommendation [skip:ci] --- specifications/SPEC-APPLICATION-SERVICE.md | 86 +++- specifications/SPEC-BUCKET-SERVICE.md | 121 +++-- specifications/SPEC-DATASET-SERVICE.md | 8 - .../SPEC-MODULE-SERVICE-TEMPLATE.md | 27 +- specifications/SPEC-QUPATH-SERVICE.md | 87 +++- specifications/SPEC_GUI_SERVICE.md | 487 +++--------------- specifications/SPEC_PLATFORM_SERVICE.md | 99 ++-- specifications/SPEC_SYSTEM_SERVICE.md | 69 ++- specifications/SPEC_WSI_SERVICE.md | 26 +- 9 files changed, 399 insertions(+), 611 deletions(-) diff --git a/specifications/SPEC-APPLICATION-SERVICE.md b/specifications/SPEC-APPLICATION-SERVICE.md index 453f6845..193b75b2 100644 --- a/specifications/SPEC-APPLICATION-SERVICE.md +++ b/specifications/SPEC-APPLICATION-SERVICE.md @@ -88,25 +88,73 @@ application/ ### 3.1 Inputs -| Input Type | Source | Code Location | -| -------------------------- | ------------- | ------------------------------------------------------------- | -| **Supported WSI Files** | CLI/GUI | `WSI_SUPPORTED_FILE_EXTENSIONS` in `constants.py` | -| **Application Version ID** | API | `Service.application_run_submit(application_version_id: str)` | -| **Input Items** | API | `Service.application_run_submit(items: list[InputItem])` | -| **Run ID** | API | `Service.application_run_download(run_id: str)` | -| **Upload Chunks** | Configuration | Constants in `_service.py` | +| Input Type | Source | Data Type/Format | Validation Rules | Business Rules | +| -------------------------- | ------------- | ---------------- | ------------------------------------------------------ | ----------------------------------------------- | +| **Supported WSI Files** | CLI/GUI | Path object | Must exist, extension in WSI_SUPPORTED_FILE_EXTENSIONS | File must be readable, format must be supported | +| **Application Version ID** | API | String | Must be valid UUID format | Must correspond to existing application version | +| **Input Items** | API | List[InputItem] | Each item must have valid metadata | Items must match application input schema | +| **Run ID** | API | String | Must be valid UUID format | Must correspond to existing application run | +| **Upload Chunks** | Configuration | Integer | Must be positive value | Configurable based on platform limits | ### 3.2 Outputs -| Output Type | Destination | Code Location | -| ---------------------- | ---------------- | ------------------------------------------------ | -| **Application Runs** | Platform API | `Service.application_run_submit()` return value | -| **Downloaded Results** | Local filesystem | `Service.application_run_download()` side effect | -| **QuPath Projects** | Local filesystem | QuPath integration when `has_qupath_extra=True` | -| **Progress Updates** | Callback/GUI | `DownloadProgress` model with computed fields | -| **Metadata Reports** | CLI/GUI | CLI commands and service methods | +| Output Type | Destination | Data Type/Format | Success Criteria | Error Conditions | +| ---------------------- | ---------------- | --------------------- | -------------------------------------------------- | --------------------------------------- | +| **Application Runs** | Platform API | ApplicationRun object | Run successfully submitted with valid ID | Platform API failure, validation errors | +| **Downloaded Results** | Local filesystem | Directory structure | All artifacts downloaded to organized directories | Network failure, permission errors | +| **QuPath Projects** | Local filesystem | .qpproj file | Valid QuPath project with input/result integration | QuPath dependency missing, file errors | +| **Progress Updates** | Callback/GUI | DownloadProgress | Real-time progress tracking with normalized values | Callback execution errors | +| **Metadata Reports** | CLI/GUI | Formatted text/JSON | Human-readable metadata display | Processing errors, missing files | + +### 3.3 Data Schemas + +**InputItem Schema:** + +```yaml +InputItem: + type: object + properties: + path: + type: string + description: File system path to WSI file + metadata: + type: object + description: Extracted WSI metadata including dimensions and format + bucket_key: + type: string + description: Cloud storage key after upload + required: [path, metadata] +``` + +**DownloadProgress Schema:** + +```yaml +DownloadProgress: + type: object + properties: + state: + type: string + enum: [INITIALIZING, CHECKING, DOWNLOADING, QUPATH_ADD_RESULTS, COMPLETED] + total_artifact_count: + type: integer + description: Total number of artifacts to download + total_artifact_index: + type: integer + description: Current artifact being processed + item_progress_normalized: + type: number + minimum: 0 + maximum: 1 + description: Progress for current item (0-1) + artifact_progress_normalized: + type: number + minimum: 0 + maximum: 1 + description: Overall progress across all artifacts (0-1) + required: [state, total_artifact_count, total_artifact_index] +``` -### 3.3 Data Flow +### 3.4 Data Flow ```mermaid graph TD @@ -371,20 +419,22 @@ Configuration is managed through environment variables with the prefix `AIGNOSTI ## 9. Implementation Details -### 9.1 Key Algorithms +### 9.1 Key Algorithms and Business Logic - **Metadata Generation Pipeline**: Multi-stage pipeline for WSI file discovery, metadata extraction, and validation - **Progress Tracking Algorithm**: Normalized progress calculation with multi-level aggregation across files and operations - **Chunked Upload Algorithm**: Memory-efficient streaming upload with integrity verification and resume capability -### 9.2 State Management +### 9.2 State Management and Data Flow - **Configuration State**: Environment-aware settings management with Pydantic validation and secure credential handling - **Runtime State**: Progress tracking state persistence for resumable operations and error recovery - **Cache Management**: Platform client caching with lazy initialization and automatic session management -### 9.3 Concurrency and Threading +### 9.3 Performance and Scalability Considerations - **Async Operations**: Asynchronous file upload/download operations with configurable concurrency limits - **Thread Safety**: Thread-safe progress tracking and state management with queue-based communication - **Resource Management**: Proper cleanup of network connections and file handles with context managers +- **Memory Efficiency**: Handle multi-gigabyte files through streaming and chunked operations +- **Scalability Patterns**: Integration with cloud storage services for horizontal scaling diff --git a/specifications/SPEC-BUCKET-SERVICE.md b/specifications/SPEC-BUCKET-SERVICE.md index 788f7435..e2f191d3 100644 --- a/specifications/SPEC-BUCKET-SERVICE.md +++ b/specifications/SPEC-BUCKET-SERVICE.md @@ -1,11 +1,11 @@ --- -itemId: SPEC-BUCKET-SERVICE +itemId: SPEC-BUCKET-SERVICE itemTitle: Bucket Module Specification -itemType: Software Item Spec -itemFulfills: SWR-BUCKET-1-1, SWR-BUCKET-1-2, SWR-BUCKET-1-3 -Module: Bucket -Layer: Domain Service -Version: 0.2.105 +itemType: Software Item Spec +itemFulfills: SWR-BUCKET-1-1, SWR-BUCKET-1-2, SWR-BUCKET-1-3 +Module: Bucket +Layer: Domain Service +Version: 0.2.105 Date: 2025-09-09 --- @@ -90,25 +90,71 @@ def read_in_chunks(): ### 3.1 Inputs -| Input Type | Source | Format/Type | Validation Rules | Code Location | -| ------------------ | ------------- | --------------- | --------------------------------------------------------- | ------------------------------------------------------ | -| Bucket Name | CLI/GUI/API | String | Must match GCS bucket naming conventions | `_service.py::Service.get_bucket_name()` method | -| Object Key/Pattern | CLI/GUI/API | String/Regex | Valid path characters, regex patterns for bulk operations | `_service.py` upload/download methods, `_cli.py` args | -| Local File Path | CLI/GUI/API | Path | Must exist for upload, valid directory for download | `_cli.py` typer Path validation, `_service.py` checks | -| Credentials | Environment | HMAC Key Pair | Required AIGNOSTICS_BUCKET_HMAC_* variables | `_settings.py::Settings` environment variable binding | -| Protocol | Configuration | String | Must be "gs" or "s3" | `_settings.py::BucketProtocol` enum validation | +| Input Type | Source | Data Type/Format | Validation Rules | Business Rules | +| ------------------ | ------------- | ---------------- | --------------------------------------------------------- | ------------------------------------------------------- | +| Bucket Name | CLI/GUI/API | String | Must match GCS bucket naming conventions | Must correspond to accessible cloud storage bucket | +| Object Key/Pattern | CLI/GUI/API | String/Regex | Valid path characters, regex patterns for bulk operations | Keys must follow cloud storage path conventions | +| Local File Path | CLI/GUI/API | Path | Must exist for upload, valid directory for download | File must be readable, directories must be writable | +| Credentials | Environment | HMAC Key Pair | Required AIGNOSTICS_BUCKET_HMAC_* variables | Keys must have appropriate bucket permissions | +| Protocol | Configuration | String | Must be "gs" or "s3" | Protocol must match configured cloud storage provider | ### 3.2 Outputs -| Output Type | Destination | Format/Type | Success Criteria | Code Location | -| ---------------- | --------------- | ---------------- | --------------------------------------------- | ------------------------------------------------------- | -| Uploaded Files | Cloud Storage | Binary/Metadata | Successful S3 PUT with ETag confirmation | `_service.py::Service.upload()` method return | -| Downloaded Files | Local Filesystem| Binary | Complete download with ETag validation | `_service.py::Service.download()` method with progress | -| Signed URLs | Client/Platform | HTTPS URL | Valid URL with correct expiration time | `_service.py::Service.create_signed_*_url()` methods | -| Progress Updates | CLI/GUI | Progress Models | Real-time byte-level progress information | `_service.py::DownloadProgress/UploadProgress` models | -| Operation Status | Logs/Console | Structured Logs | Success/failure with detailed error messages | `_cli.py` console output, `_service.py` logger calls | +| Output Type | Destination | Data Type/Format | Success Criteria | Error Conditions | +| ---------------- | ---------------- | ---------------- | --------------------------------------------- | ------------------------------------------- | +| Uploaded Files | Cloud Storage | Binary/Metadata | Successful S3 PUT with ETag confirmation | Network failure, permission errors | +| Downloaded Files | Local Filesystem | Binary | Complete download with ETag validation | Disk space issues, permission errors | +| Signed URLs | Client/Platform | HTTPS URL | Valid URL with correct expiration time | Credential errors, invalid object keys | +| Progress Updates | CLI/GUI | Progress Models | Real-time byte-level progress information | Callback execution errors | +| Operation Status | Logs/Console | Structured Logs | Success/failure with detailed error messages | Logging system failures | + +### 3.3 Data Schemas + +**DownloadProgress Schema:** + +```yaml +DownloadProgress: + type: object + properties: + total_bytes: + type: integer + description: Total bytes to download + downloaded_bytes: + type: integer + description: Bytes downloaded so far + current_file: + type: string + description: Current file being downloaded + progress_percentage: + type: number + minimum: 0 + maximum: 100 + description: Download progress as percentage + required: [total_bytes, downloaded_bytes] +``` + +**UploadProgress Schema:** + +```yaml +UploadProgress: + type: object + properties: + total_bytes: + type: integer + description: Total bytes to upload + uploaded_bytes: + type: integer + description: Bytes uploaded so far + current_file: + type: string + description: Current file being uploaded + upload_speed: + type: number + description: Upload speed in bytes per second + required: [total_bytes, uploaded_bytes] +``` -### 3.3 Data Flow +### 3.4 Data Flow ```mermaid graph LR @@ -305,39 +351,26 @@ uvx aignostics bucket [subcommand] [options] --- -## 9. Testing and Quality Assurance - -### 9.1 Testing Strategy - -- **Unit Tests**: Mock S3 client for isolated service testing, validate all public methods -- **Integration Tests**: Real cloud storage operations in test environment -- **Performance Tests**: Large file upload/download benchmarks, concurrent operation testing -- **Security Tests**: Credential handling validation, input sanitization verification - -### 9.2 Quality Metrics - -- **Code Coverage**: Minimum 80% test coverage for service layer -- **Performance Benchmarks**: <30s for 1GB file operations, <5s for signed URL generation -- **Reliability Targets**: 99.9% operation success rate, <1% data corruption tolerance - ---- - -## 10. Implementation Details +## 9. Implementation Details -### 10.1 Key Algorithms +### 9.1 Key Algorithms and Business Logic - **Chunked Transfer**: Adaptive chunk sizing based on operation type (1MB upload, 10MB download, 100MB ETag) - **ETag Caching**: MD5-based content comparison to avoid redundant downloads - **Progress Calculation**: Byte-level progress tracking with transfer speed estimation +- **Pattern Matching**: Regex-based object filtering for bulk operations and content discovery -### 10.2 State Management +### 9.2 State Management and Data Flow - **Configuration State**: Settings cached from environment variables with lazy loading - **Runtime State**: Progress models maintain operation state with real-time updates - **Cache Management**: ETag-based file validation cache for efficient re-download detection +- **Session Management**: S3 client connection pooling and automatic retry mechanisms -### 10.3 Concurrency and Threading +### 9.3 Performance and Scalability Considerations -- **Async Operations**: Generator-based progress callbacks for non-blocking UI updates -- **Thread Safety**: Immutable progress models, thread-safe logging configuration +- **Memory Efficiency**: Streaming operations for large files with configurable chunk sizes +- **Network Optimization**: Connection pooling, retry mechanisms, and bandwidth throttling +- **Concurrent Operations**: Thread-safe progress tracking and parallel transfer support - **Resource Management**: Proper cleanup of S3 client connections and file handles +- **Scalability Patterns**: Support for high-throughput operations with memory constraints diff --git a/specifications/SPEC-DATASET-SERVICE.md b/specifications/SPEC-DATASET-SERVICE.md index 3ad71761..7305dda6 100644 --- a/specifications/SPEC-DATASET-SERVICE.md +++ b/specifications/SPEC-DATASET-SERVICE.md @@ -9,14 +9,6 @@ Version: 0.2.105 Date: 2025-09-11 --- -## Documentation Guidelin| Parameter | Type | Default | Description | Required | - -| -------------------- | ---- | ---------------------------------------------------------------- | --------------------------------- | -------- | -| `target_layout` | str | `%collection_id/%PatientID/%StudyInstanceUID/%Modality_%SeriesInstanceUID/` | Directory layout template | No | -| `portal_url` | str | `https://portal.imaging.datacommons.cancer.gov/explore/` | IDC portal base URL | No | -| `example_dataset_id` | str | `1.3.6.1.4.1.5962.99.1.1069745200.1645485340.1637452317744.2.0` | Example dataset for testing | No | -| `path_length_max` | int | 260 | Maximum path length (Windows) | No |# Code in Specifications - Best Practices - ## 1. Description ### 1.1 Purpose diff --git a/specifications/SPEC-MODULE-SERVICE-TEMPLATE.md b/specifications/SPEC-MODULE-SERVICE-TEMPLATE.md index 90e66c03..2d3e6b51 100644 --- a/specifications/SPEC-MODULE-SERVICE-TEMPLATE.md +++ b/specifications/SPEC-MODULE-SERVICE-TEMPLATE.md @@ -9,7 +9,32 @@ Version: [VERSION] _(e.g., 0.2.105)_ Date: [DATE] --- -## Documentation Guidelines +## Documentation Guidelines [DO NOT ADD] + +### Code in Specifications - Best Practices + +**INCLUDE Code When:** + +- ✅ Public API signatures (stable contracts) +- ✅ Data structure schemas for inputs/outputs +- ✅ Configuration parameter definitions +- ✅ Error type hierarchies + +**AVOID Code When:** + +- ❌ Internal implementation details +- ❌ Private methods or functions +- ❌ Complete code blocks or algorithms +- ❌ Version-specific dependency details + +**Preferred Approaches:** + +- 📋 Reference interfaces by name and purpose +- 📋 Use schemas (JSON Schema, OpenAPI) for data structures +- 📋 Link to auto-generated documentation for details +- 📋 Focus on behavior and contracts, not implementation + +--- ## 1. Description diff --git a/specifications/SPEC-QUPATH-SERVICE.md b/specifications/SPEC-QUPATH-SERVICE.md index 00dac2a9..8c659579 100644 --- a/specifications/SPEC-QUPATH-SERVICE.md +++ b/specifications/SPEC-QUPATH-SERVICE.md @@ -1,11 +1,11 @@ --- -itemId: SPEC-QUPATH-SERVICE +itemId: SPEC-QUPATH-SERVICE itemTitle: QuPath Module Specification -itemType: Software Item Spec +itemType: Software Item Spec itemFulfills: SWR-VISUALIZATION-1-1, SWR-VISUALIZATION-1-2, SWR-VISUALIZATION-1-3 -Module: QuPath -Layer: Domain Service -Version: 0.2.105 +Module: QuPath +Layer: Domain Service +Version: 0.2.105 Date: 2025-09-03 --- @@ -73,14 +73,16 @@ qupath/ ### 2.2 Key Components -| Component | Type | Purpose | Public API | -| --------------- | ------ | ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- | -| `Service` | Class | Core QuPath management, installation, and project operations | `install_qupath()`, `uninstall_qupath()`, `health()`, `add()`, `annotate()`, `inspect()` | -| `QuPathVersion` | Model | Version information storage and validation | `version`, `build_time`, `commit_tag` properties | -| `QuPathProject` | Model | Project information and image metadata | `uri`, `version`, `images` properties | -| `QuPathImage` | Model | Individual image information in projects | Image metadata, hierarchy, and file path information | -| `CLI` | Module | Command-line interface with Typer commands | `install`, `uninstall`, `launch`, `add`, `annotate`, `inspect`, `processes`, `terminate` commands | -| `GUI` | Module | Web-based interface using GUI framework | Interactive QuPath management dashboard | +| Component | Type | Purpose | Public API | +| ------------------ | ------ | ------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------- | +| `Service` | Class | Core QuPath management, installation, and project operations | `install_qupath()`, `uninstall_qupath()`, `health()`, `add()`, `annotate()`, `inspect()` | +| `QuPathVersion` | Model | Version information storage and validation | `version`, `build_time`, `commit_tag` properties | +| `QuPathProject` | Model | Project information and image metadata | `uri`, `version`, `images` properties | +| `QuPathImage` | Model | Individual image information in projects | Image metadata, hierarchy, and file path information | +| `AddProgress` | Model | Progress tracking for image addition operations | `status`, `image_count`, `image_index`, `progress_normalized` properties | +| `AnnotateProgress` | Model | Progress tracking for annotation import operations | `status`, `annotation_count`, `annotation_index`, `progress_normalized` properties | +| `CLI` | Module | Command-line interface with Typer commands | `install`, `launch`, `processes`, `terminate`, `uninstall`, `add`, `annotate`, `inspect`, `run-script` commands | +| `GUI` | Module | Web-based interface using GUI framework | Interactive QuPath management dashboard | ### 2.3 Design Patterns @@ -117,7 +119,49 @@ qupath/ | Annotation Import Results | CLI/API | Integer Count | Number of annotations successfully imported to image | | Project Information | CLI/API | QuPathProject Model | Complete project metadata including images and hierarchy info | -### 3.3 Data Flow +### 3.3 Data Schemas + +**QuPath Version Schema:** + +```yaml +QuPathVersion: + type: object + properties: + version: + type: string + description: QuPath version string + pattern: "^[0-9]+\.[0-9]+\.[0-9]+.*$" + build_time: + type: string + description: Build timestamp + format: date-time + commit_tag: + type: string + description: Git commit tag + pattern: "^[a-f0-9]+$" +``` + +**QuPath Project Schema:** + +```yaml +QuPathProject: + type: object + properties: + uri: + type: string + description: Project file URI + format: uri + version: + type: string + description: QuPath version used + images: + type: array + items: + $ref: "#/definitions/QuPathImage" + description: List of images in project +``` + +### 3.4 Data Flow ```mermaid graph LR @@ -153,8 +197,9 @@ class Service(BaseService): reinstall: bool = True, platform_system: str | None = None, platform_machine: str | None = None, - download_progress: Callable[[Path, int, int], None] | None = None, - extract_progress: Callable[[Path, int], None] | None = None, + download_progress: Callable | None = None, + extract_progress: Callable | None = None, + progress_queue: queue.Queue[InstallProgress] | None = None, ) -> Path: ... @staticmethod @@ -197,10 +242,10 @@ uvx aignostics qupath [subcommand] [options] **Available Commands:** - `install`: Download and install QuPath with version and platform options -- `uninstall`: Remove QuPath installation with platform-specific cleanup - `launch`: Launch QuPath application with optional project and script parameters - `processes`: List running QuPath processes with detailed information - `terminate`: Terminate all running QuPath processes +- `uninstall`: Remove QuPath installation with platform-specific cleanup - `add`: Add images to QuPath project (creates project if needed) - `annotate`: Import GeoJSON annotations to specific image in project - `inspect`: Examine project structure and metadata @@ -260,7 +305,7 @@ uvx aignostics qupath terminate | ----------------- | -------------------------------------------------- | --------------------------------------------- | | Utils Module | Logging, base service class, and common utilities | `BaseService`, `get_logger()`, `console` | | CLI Module | Command registration and CLI framework integration | CLI router registration and command discovery | -| GUI Module | Web interface framework and component libraries | Streamlit components and page registration | +| GUI Module | Web interface framework and component libraries | NiceGUI components and page registration | ### 5.2 External Dependencies @@ -346,20 +391,20 @@ uvx aignostics qupath terminate ## 9. Implementation Details -### 10.1 Key Algorithms +### 9.1 Key Algorithms and Business Logic - **Platform Detection**: Automatic detection of OS and architecture for QuPath binary selection - **Chunked Download**: Efficient download of large QuPath archives with progress tracking - **Streaming JSON Processing**: Memory-efficient processing of large annotation datasets using ijson - **Process Management**: Safe execution and monitoring of QuPath subprocess with timeout handling -### 10.2 State Management +### 9.2 State Management and Data Flow - **Configuration State**: Module settings stored using Pydantic settings with environment variable override - **Runtime State**: QuPath process information tracked in memory with psutil integration - **Installation State**: QuPath installation status persisted in user data directory -### 10.3 Concurrency and Threading +### 9.3 Performance and Scalability Considerations - **Async Operations**: Download and extraction operations run in background with progress callbacks - **Thread Safety**: All service methods are thread-safe for concurrent access diff --git a/specifications/SPEC_GUI_SERVICE.md b/specifications/SPEC_GUI_SERVICE.md index cf257d5a..5282a67b 100644 --- a/specifications/SPEC_GUI_SERVICE.md +++ b/specifications/SPEC_GUI_SERVICE.md @@ -69,7 +69,7 @@ gui/ | `PageBuilder` | Class | Theme and static asset registration | BasePageBuilder pattern implementation | Utils Module | | `ErrorPageBuilder` | Class | Error page handling and fallbacks | Error scenario page registration | Utils Module | -_Note: For detailed implementation, refer to the source code in the `src/aignostics/gui/` directory._ +For detailed implementation, refer to the source code in the `src/aignostics/gui/` directory. ### 2.3 Design Patterns @@ -78,120 +78,7 @@ _Note: For detailed implementation, refer to the source code in the `src/aignost - **Factory Pattern**: Theme application with configurable styling and brand asset loading - **Observer Pattern**: Health monitoring integration with automatic UI updates and status propagation - **Conditional Loading Pattern**: Optional dependency detection and graceful degradation when GUI frameworks unavailable - -### 2.3 Design Patterns - -- **PageBuilder Pattern**: Abstract base class pattern for module GUI registration with standardized `register_pages()` method -- **Conditional Loading Pattern**: Feature detection and graceful degradation using `find_spec()` checks -- **Context Manager Pattern**: Frame component uses context manager for consistent layout application -- **Auto-Discovery Pattern**: Automatic detection and registration of module GUI components - -### 2.4 GUI Module Architecture Diagram - -```mermaid -graph TB - subgraph "Core GUI Module (src/aignostics/gui/)" - GUI_INIT["__init__.py
• Conditional loading
• Auto-discovery
• Exports: frame, theme, ErrorPageBuilder, PageBuilder, HEALTH_UPDATE_INTERVAL"] - GUI_FRAME["_frame.py
• frame() context manager
• Health monitoring
• User authentication
• HEALTH_UPDATE_INTERVAL = 30
• USERINFO_UPDATE_INTERVAL = 3600"] - GUI_THEME["_theme.py
• theme() function
• CSS application
• Font loading
• PageBuilder class"] - GUI_ERROR["_error.py
• Error PageBuilder
• Exception handling
• Fallback pages"] - GUI_ASSETS["assets/
• cabin-v27-latin-regular.woff2
• cat.lottie
• logo.png"] - end - - subgraph "Utils Module" - UTILS_GUI["utils/_gui.py
• BasePageBuilder (abstract)
• gui_register_pages()
• gui_run()
• locate_subclasses()"] - end - - subgraph "Module GUI Implementations" - APP_GUI["application/_gui/
• _page_builder.py (PageBuilder)
• _frame.py (uses GUI frame)
• Multiple page files
• Routes: /, /application/{id}
• Assets: /application_assets/*"] - BUCKET_GUI["bucket/_gui.py
• PageBuilder class
• Route: /bucket
• Uses GUI frame
• Uses relative import for BasePageBuilder
• Assets: /bucket_assets/*"] - DATASET_GUI["dataset/_gui.py
• PageBuilder class
• Route: /dataset
• Uses GUI frame
• Uses relative import for BasePageBuilder
• Assets: /dataset_assets/*"] - NOTEBOOK_GUI["notebook/_gui.py
• PageBuilder class
• Route: /notebook
• Uses GUI frame + theme
• Assets: /notebook_assets/*"] - QUPATH_GUI["qupath/_gui.py
• PageBuilder class
• Route: /qupath
• Uses GUI frame
• Assets: /qupath_assets/*"] - SYSTEM_GUI["system/_gui.py
• PageBuilder class
• Route: /system
• Uses GUI frame
• Uses relative import for BasePageBuilder
• Assets: /system_assets/*"] - WSI_GUI["wsi/_gui.py
• PageBuilder class
• API endpoints: /thumbnail, /tiff
• No GUI frame usage
• Assets: /wsi_assets/*"] - end - - subgraph "CLI Integration" - CLI["cli.py
• launchpad command
• Conditional loading
• Calls gui_run()"] - end - - %% Core GUI internal dependencies - GUI_INIT --> GUI_FRAME - GUI_INIT --> GUI_THEME - GUI_INIT --> GUI_ERROR - GUI_THEME --> GUI_ASSETS - - %% Module dependencies on GUI core - APP_GUI --> GUI_FRAME - BUCKET_GUI --> GUI_FRAME - DATASET_GUI --> GUI_FRAME - NOTEBOOK_GUI --> GUI_FRAME - NOTEBOOK_GUI --> GUI_THEME - QUPATH_GUI --> GUI_FRAME - SYSTEM_GUI --> GUI_FRAME - %% WSI_GUI does NOT import from GUI (only provides API endpoints) - - %% Module dependencies on Utils (different import patterns) - APP_GUI --> UTILS_GUI - NOTEBOOK_GUI --> UTILS_GUI - QUPATH_GUI --> UTILS_GUI - WSI_GUI --> UTILS_GUI - %% These use relative imports from ..utils - BUCKET_GUI -.-> UTILS_GUI - DATASET_GUI -.-> UTILS_GUI - SYSTEM_GUI -.-> UTILS_GUI - - %% CLI integration - CLI --> UTILS_GUI - UTILS_GUI --> GUI_INIT - - %% Auto-discovery flow (utils/_gui.py finds all PageBuilder subclasses) - UTILS_GUI -.-> APP_GUI - UTILS_GUI -.-> BUCKET_GUI - UTILS_GUI -.-> DATASET_GUI - UTILS_GUI -.-> NOTEBOOK_GUI - UTILS_GUI -.-> QUPATH_GUI - UTILS_GUI -.-> SYSTEM_GUI - UTILS_GUI -.-> WSI_GUI - - %% Styling - classDef coreModule fill:#e1f5fe,stroke:#01579b,stroke-width:2px - classDef moduleGui fill:#f3e5f5,stroke:#4a148c,stroke-width:2px - classDef utils fill:#e8f5e8,stroke:#1b5e20,stroke-width:2px - classDef assets fill:#fff3e0,stroke:#e65100,stroke-width:2px - classDef cli fill:#fce4ec,stroke:#880e4f,stroke-width:2px - - class GUI_INIT,GUI_FRAME,GUI_THEME,GUI_ERROR,GUI_ASSETS coreModule - class APP_GUI,BUCKET_GUI,DATASET_GUI,NOTEBOOK_GUI,QUPATH_GUI,SYSTEM_GUI,WSI_GUI moduleGui - class UTILS_GUI utils - class CLI cli -``` - -**Key Findings from Code Verification:** - -1. **Core GUI Module**: Correctly exports `frame`, `theme`, `ErrorPageBuilder`, `PageBuilder`, and `HEALTH_UPDATE_INTERVAL` - -2. **Import Patterns**: - - - **Direct imports**: `application`, `notebook`, `qupath`, `wsi` use `from aignostics.utils import BasePageBuilder` - - **Relative imports**: `bucket`, `dataset`, `system` use `from ..utils import BasePageBuilder` - - **Frame usage**: All modules except `wsi` import and use `from aignostics.gui import frame` - -3. **WSI Module Special Case**: Only provides API endpoints (`/thumbnail`, `/tiff`) and doesn't use the frame component - -4. **Application Module Structure**: Uses directory structure with multiple page files, unlike other modules with single `_gui.py` files - -5. **Asset Management**: All modules follow the `/module_name_assets/*` pattern consistently - -6. **Auto-Discovery**: Implemented through `locate_subclasses(BasePageBuilder)` in `utils/_gui.py`**Key Relationships:** - -- **Solid arrows**: Direct imports and dependencies -- **Dotted arrows**: Auto-discovery mechanism (utils/\_gui.py finds all PageBuilder subclasses) -- **Core GUI Module**: Provides framework components (frame, theme, error handling) -- **Module GUI Implementations**: Each module has its own GUI implementation following PageBuilder pattern -- **Asset Management**: Each module serves static assets through module-specific routes -- **CLI Integration**: Conditional launchpad command integrates with GUI framework +- **Auto-Discovery Pattern**: Automatic detection and registration of module GUI components using `locate_subclasses()` --- @@ -222,69 +109,53 @@ graph TB **Frame Configuration Schema:** ```yaml -# Source: Based on frame() function signature and usage patterns +# Frame function parameters frame_config: type: object required: [navigation_title] properties: navigation_title: type: string - description: Display title for page navigation - navigation_icon: - type: string - nullable: true - description: Icon identifier for navigation display - navigation_icon_color: + description: Title displayed in navigation bar + validation: Non-empty string + icon: type: string - nullable: true - description: Color specification for navigation icon + description: Icon identifier for navigation + validation: Valid NiceGUI icon name or null left_sidebar: type: boolean + description: Enable left sidebar display default: false - description: Enable left sidebar in page layout ``` **Theme Configuration Schema:** ```yaml -# Source: Based on theme() function color and styling definitions +# Theme styling configuration theme_config: type: object properties: colors: type: object - properties: - primary: - type: string - description: Primary brand color (#1C1242) - secondary: - type: string - description: Secondary brand color (#B9B1DF) - accent: - type: string - description: Accent color (#111B1E) - positive: - type: string - description: Success/positive color (#0CA57B) - negative: - type: string - description: Error/negative color (#D4313C) - # Additional brand colors: info, warning, brand_white, brand_background_light, brand_logo + description: Aignostics brand color scheme fonts: type: object - properties: - cabin: - type: string - description: Custom font family definition + description: Custom font definitions + css_overrides: + type: string + description: Additional CSS styling ``` -_Note: Complete schemas available in implementation docstrings and type hints._ +Actual schemas may be defined in OpenAPI specifications or JSON Schema files. ### 3.4 Data Flow -``` -Module Registration → PageBuilder Discovery → Frame Layout → Theme Application → Asset Serving -Health Monitoring → Status Updates → Navigation Display → User Interface → Error Handling +```mermaid +graph LR + A[Module Registration] --> B[PageBuilder Discovery] --> C[Frame Layout] + C --> D[Theme Application] --> E[Asset Serving] + F[Health Monitoring] --> G[Status Updates] --> H[Navigation Display] + I[User Interface] --> J[Error Handling] --> K[Fallback Pages] ``` --- @@ -293,230 +164,61 @@ Health Monitoring → Status Updates → Navigation Display → User Interface ### 4.1 Public API -#### Core Layout Interface +#### Core Service Interface **Frame Context Manager**: `frame()` - **Purpose**: Provides standardized page layout with navigation, branding, and health monitoring integration -- **Key Capabilities**: - - Context manager for consistent page structure across all module interfaces - - Navigation title and icon configuration with tooltip support - - Optional left sidebar display for module-specific navigation - - Integrated health monitoring with automatic status updates - - User authentication status display and profile integration - -**Input/Output Contracts**: - -- **Initialization**: Navigation title (required), optional icon and layout configuration -- **Layout Output**: Complete HTML page structure with navigation, sidebar, and content areas -- **Error Handling**: Graceful fallback when NiceGUI dependencies unavailable - -#### Theme Management Interface +- **Key Methods**: + - `frame(navigation_title: str, icon: str | None = None, left_sidebar: bool = False)`: Creates consistent page layout structure +- **Input/Output Contracts**: + - **Input Types**: Navigation title (required string), optional icon and layout configuration + - **Output Types**: HTML page structure with navigation, sidebar, and content areas + - **Error Conditions**: Graceful fallback when NiceGUI dependencies unavailable **Theme Application Function**: `theme()` - **Purpose**: Applies consistent Aignostics branding and styling across all interfaces -- **Capabilities**: CSS color scheme application, custom font loading, responsive design patterns - -#### PageBuilder Pattern Interface +- **Key Methods**: + - `theme() -> None`: Applies CSS color scheme, custom fonts, and responsive design patterns **PageBuilder Classes**: `PageBuilder`, `ErrorPageBuilder` - **Purpose**: Standard interface for module-specific GUI component registration and static asset management -- **Registration Pattern**: Abstract base requiring implementation of `register_pages()` method for route and asset registration - -_Note: For detailed method signatures, refer to the module's `__init__.py` and implementation files._ - -- Consistent styling framework - -### 3.4 Error Handling Component - -```python -class PageBuilder(BasePageBuilder): - """Page builder for error scenarios.""" -``` - -**Implemented Features**: - -- Error page framework -- Graceful degradation patterns - -### 3.5 CLI Integration - -```python -# In cli.py -if find_spec("nicegui") and find_spec("webview") and not __is_running_in_container__: - @cli.command() - def launchpad() -> None: - """Open Aignostics Launchpad, the graphical user interface.""" - from .utils import gui_run - gui_run(native=True, with_api=False, title="Aignostics Launchpad", icon="🔬") -``` - -## 4. Module GUI Integration Patterns - -### 4.1 Standard Module GUI Structure - -Each module follows this verified pattern: - -```python -# Example from wsi/_gui.py -class PageBuilder(BasePageBuilder): - @staticmethod - def register_pages() -> None: - from nicegui import app - - # Static assets registration - app.add_static_files("/wsi_assets", Path(__file__).parent / "assets") - - # API endpoint registration - @app.get("/thumbnail") - def thumbnail(source: str) -> Response: - """Serve a thumbnail for a given source reference.""" - from fastapi import Response - from fastapi.responses import RedirectResponse - - try: - return Response( - content=Service().get_thumbnail_bytes(Path(source)), - media_type="image/png" - ) - except ValueError: - logger.warning("Error generating thumbnail on bad request") - return RedirectResponse("/wsi_assets/fallback.png") - except RuntimeError: - logger.exception("Internal server error when generating thumbnail") - return RedirectResponse("/wsi_assets/fallback.png") -``` - -### 4.2 Error Handling Pattern +- **Key Methods**: + - `register_pages() -> None`: Abstract method for route and asset registration -Consistent error handling across modules: +### 4.2 CLI Interface (if applicable) -- `ValueError` → Client errors (400-level) → Warning logs → Fallback redirect -- `RuntimeError` → Server errors (500-level) → Exception logs → Fallback redirect -- Fallback assets served from module-specific asset directories +**Command Structure:** -### 4.3 Asset Management Pattern - -```python -# Consistent naming convention -app.add_static_files("/module_name_assets", Path(__file__).parent / "assets") - -# Example implementations found: -# /wsi_assets/ - WSI module assets -# /qupath_assets/ - QuPath module assets (with Lottie animations) +```bash +uvx aignostics launchpad ``` -### 4.4 Advanced Page Creation - -Example from QuPath module showing full page implementation: - -```python -@ui.page("/qupath") -async def page_index() -> None: - """QuPath Extension.""" - with frame("QuPath Extension", left_sidebar=False): - # Page content implementation - with ui.card(): - install_info = ui.label("QuPath installation status...") - - # Interactive components - install_button = ui.button( - "Install" if not version else "Reinstall", - on_click=install_qupath, - icon="install_desktop", - ) - - # Lottie animations for visual feedback - ui.html(f'' - f'') -``` - -### 4.5 Creating New GUI Views - -#### 4.5.1 Module GUI File Structure +**Available Commands:** -To add GUI functionality to a new service module: +| Command | Purpose | Input Requirements | Output Format | +| ----------- | ------------------------------------- | ------------------ | --------------------- | +| `launchpad` | Open graphical user interface desktop | None | Native desktop window | -1. **Create \_gui.py file** in your module directory: +**Common Options:** -```python -# filepath: src/aignostics/your_module/_gui.py -"""Your Module GUI integration.""" +- `--help`: Display command help +- Conditional availability based on NiceGUI and WebView dependencies -from __future__ import annotations -from pathlib import Path -from typing import TYPE_CHECKING +### 4.3 HTTP/Web Interface (if applicable) -from aignostics.utils import BasePageBuilder, get_logger +**Endpoint Structure:** -if TYPE_CHECKING: - from fastapi import Response +| Method | Endpoint | Purpose | Request Format | Response Format | +| ------ | ----------------------- | --------------------- | -------------- | --------------- | +| `GET` | `/` | Main application page | None | HTML page | +| `GET` | `/module_name_assets/*` | Static asset serving | File path | Binary content | +| `GET` | `/module_name/*` | Module-specific pages | None | HTML page | -from ._service import Service - -logger = get_logger(__name__) - -class PageBuilder(BasePageBuilder): - @staticmethod - def register_pages() -> None: - from nicegui import app - - # Register static assets - app.add_static_files("/your_module_assets", Path(__file__).parent / "assets") - - # Register API endpoints - @app.get("/your_module/endpoint") - def your_endpoint(param: str) -> Response: - """Your endpoint documentation.""" - # Implementation here -``` - -#### 4.5.2 Using the Frame Component - -The `frame` component provides consistent layout across all GUI views: - -```python -from aignostics.gui import frame - -# Create a new page with consistent layout -@app.page("/your_module") -def your_page(): - """Your module's main page.""" - with frame( - navigation_title="Your Module", - navigation_icon="your_icon", - navigation_icon_color="primary", - navigation_icon_tooltip="Your module description", - left_sidebar=True # Enable sidebar for complex layouts - ): - # Your page content here - ui.label("Your module content") - - # Service integration - with ui.card(): - ui.label("Service Status") - service_health = YourService().health() - ui.label(f"Status: {service_health.status}") -``` - -## 5. Configuration and Settings - -### 5.1 Configuration Parameters - -| Parameter | Type | Default | Description | Required | -| -------------------------- | ---- | ------- | ------------------------------------- | -------- | -| `HEALTH_UPDATE_INTERVAL` | int | 30 | Health check frequency (seconds) | Yes | -| `USERINFO_UPDATE_INTERVAL` | int | 3600 | User info refresh frequency (seconds) | Yes | - -### 5.2 Environment Variables - -| Variable | Purpose | Example Value | -| ----------------------------- | -------------------------------------- | -------------- | -| `__is_running_in_container__` | Container detection for feature gating | `true`/`false` | +**Authentication**: User authentication status display integrated in navigation +**Error Responses**: Standardized error pages with fallback mechanisms --- @@ -541,7 +243,7 @@ def your_page(): | `webview` | ^4.0 | Native desktop application support | Optional | Web browser launch only | | `uvicorn` | ^0.23 | ASGI server for development | Optional | Development server unavailable | -_Note: For exact version requirements, refer to `pyproject.toml` and dependency lock files._ +For exact version requirements, refer to `pyproject.toml` and dependency lock files. ### 5.3 Integration Points @@ -556,19 +258,16 @@ _Note: For exact version requirements, refer to `pyproject.toml` and dependency ### 6.1 Configuration Parameters -| Parameter | Type | Default | Description | Required | -| -------------------------- | ----- | --------- | ------------------------------------- | -------- | -| `HEALTH_UPDATE_INTERVAL` | `int` | `30` | Health check frequency (seconds) | No | -| `USERINFO_UPDATE_INTERVAL` | `int` | `3600` | User info refresh frequency (seconds) | No | -| `PROFILE_EDIT_URL` | `str` | Hardcoded | Platform profile management URL | No | +| Parameter | Type | Default | Description | Required | +| -------------------------- | ----- | ------- | ------------------------------------- | -------- | +| `HEALTH_UPDATE_INTERVAL` | `int` | `30` | Health check frequency (seconds) | No | +| `USERINFO_UPDATE_INTERVAL` | `int` | `3600` | User info refresh frequency (seconds) | No | ### 6.2 Environment Variables | Variable | Purpose | Example Value | | ----------------------------- | -------------------------------------- | ------------------ | | `__is_running_in_container__` | Container detection for feature gating | `"true"`/`"false"` | -| `NICEGUI_TAILWIND_ENABLED` | Enable/disable Tailwind CSS support | `"true"` | -| `NICEGUI_DEBUG` | Enable debug mode for development | `"true"` | --- @@ -603,34 +302,25 @@ _Note: For exact version requirements, refer to `pyproject.toml` and dependency ### 8.1 Data Protection - **Authentication**: Secure display of user authentication status without exposing sensitive data -- **Asset Serving**: Restricted to module-specific directories, path validation prevents directory traversal -- **Input Sanitization**: Navigation titles and parameters sanitized for HTML output +- **Data Encryption**: In-transit encryption through HTTPS for web interface +- **Access Control**: Module-based permission system for GUI component access -### 8.2 Security Measures +### 8.2 Security Measures [Optional] -- **Path Validation**: Static asset paths validated and restricted to safe directories -- **Import Safety**: TYPE_CHECKING guards prevent execution of GUI code during type checking -- **Container Detection**: Feature gating based on environment prevents unauthorized access to desktop features +- **Input Sanitization**: Navigation titles and parameters sanitized for HTML output +- **Secret Management**: No secrets stored in GUI module, authentication handled by Platform module +- **Audit Logging**: Security events logged through standard logging framework --- ## 9. Implementation Details -### 9.1 Key Algorithms +### 9.1 Key Algorithms and Business Logic - **Auto-Discovery Algorithm**: Uses `locate_subclasses()` to find all `BasePageBuilder` implementations across modules and automatically register their pages - **Conditional Loading Algorithm**: Uses `find_spec()` to detect available dependencies and conditionally load features - **Theme Application Algorithm**: CSS injection and font loading with fallback mechanisms for consistent styling -### 9.2 State Management - -### 9.1 Key Algorithms and Business Logic - -- **Module Discovery**: Automatic detection and registration of PageBuilder implementations from all SDK modules using reflection-based service discovery -- **Conditional Loading**: Feature detection using `find_spec()` to gracefully handle optional dependencies like WebView and NiceGUI -- **Theme Application**: CSS injection and brand asset loading with fallback mechanisms for consistent styling across all interfaces -- **Health Integration**: Periodic health status updates with configurable intervals and automatic UI refresh coordination - ### 9.2 State Management and Data Flow - **State Type**: Stateful GUI application with session-based configuration and persistent theme settings @@ -651,9 +341,9 @@ _Note: For exact version requirements, refer to `pyproject.toml` and dependency ### Verification and Updates -**Last Verified**: September 11, 2025 +**Last Verified**: September 15, 2025 **Verification Method**: Code review against implementation in `src/aignostics/gui/` and template compliance check -**Next Review Date**: December 11, 2025 +**Next Review Date**: December 15, 2025 ### Change Management @@ -666,46 +356,3 @@ _Note: For exact version requirements, refer to `pyproject.toml` and dependency **Implementation**: See `src/aignostics/gui/` for current implementation **Tests**: See `tests/aignostics/gui/` for usage examples and verification **API Documentation**: Auto-generated from frame and theme function docstrings - -- **Theme Application**: CSS injection and font loading through NiceGUI's head HTML mechanism - -### 9.2 State Management - -- **Configuration State**: Health and user info update intervals stored as module constants -- **Runtime State**: Health status and user authentication cached in UI storage and updated via timers -- **Cache Management**: Static assets cached by FastAPI, UI state refreshed via NiceGUI's refreshable decorator - -### 9.3 Concurrency and Threading - -- **Async Operations**: Health monitoring and user info updates use NiceGUI timer callbacks -- **Thread Safety**: GUI components operate on main thread, timer callbacks handle concurrent updates -- **Resource Management**: Static file serving handled by FastAPI's async infrastructure - -### 9.4 Module Integration Pattern - -Each module implements GUI functionality using the standard pattern: - -```python -class PageBuilder(BasePageBuilder): - @staticmethod - def register_pages() -> None: - """Register module-specific pages and assets.""" - from nicegui import app # Lazy import to avoid circular dependencies - - # Static asset registration - app.add_static_files("/module_name_assets", Path(__file__).parent / "assets") - - # Page and endpoint registration - @app.page("/module_name") - def module_page(): - with frame(navigation_title="Module Name"): - # Module-specific UI implementation - pass -``` - -**Standard Conventions**: - -- Static assets: `/module_name_assets/` URL pattern -- Page routes: `/module_name/page_name` URL pattern -- Error handling: `ValueError` for client errors, `RuntimeError` for server errors -- Required assets: `fallback.png` for error scenarios diff --git a/specifications/SPEC_PLATFORM_SERVICE.md b/specifications/SPEC_PLATFORM_SERVICE.md index 4933a680..d62a002b 100644 --- a/specifications/SPEC_PLATFORM_SERVICE.md +++ b/specifications/SPEC_PLATFORM_SERVICE.md @@ -115,7 +115,59 @@ platform/ | Health Status | Monitoring systems | Health object | Accurate service and dependency status | `_service.py::Service.health()` method | | Downloaded Files | Local filesystem | Binary/structured data | Verified checksums and complete downloads | `_utils.py` download functions and `ApplicationRun` | -### 3.3 Data Flow +### 3.3 Data Schemas + +**Authentication Token Schema:** + +```yaml +JWTToken: + type: object + properties: + access_token: + type: string + description: JWT access token for API authentication + token_type: + type: string + enum: ["Bearer"] + description: Token type for authorization header + expires_in: + type: integer + description: Token expiration time in seconds + refresh_token: + type: string + description: Long-lived token for access token renewal + required: [access_token, token_type, expires_in] +``` + +**User Information Schema:** + +```yaml +UserInfo: + type: object + properties: + user: + type: object + properties: + id: { type: string } + name: { type: string } + email: { type: string } + picture: { type: string, nullable: true } + organization: + type: object + properties: + id: { type: string } + name: { type: string, nullable: true } + role: + type: string + description: User role within organization + token: + type: object + properties: + expires_in: { type: integer } + required: [user, organization, role, token] +``` + +### 3.4 Data Flow ```mermaid graph TD @@ -431,60 +483,23 @@ The Platform module provides foundational services but does not directly expose --- -## 9. Testing and Quality Assurance - -### 9.1 Testing Strategy - -- **Unit Tests**: Pytest-based testing with mocked external dependencies covering authentication, CLI, settings, and resource modules -- **Integration Tests**: Scheduled end-to-end tests with real API calls marked with `@pytest.mark.scheduled` and configurable timeouts -- **Mock Testing**: External services mocked using `unittest.mock` including OAuth flows, API clients, and file operations -- **Error Scenario Testing**: Comprehensive failure mode testing including network errors, authentication failures, and invalid configurations - -### 9.2 Quality Metrics - -- **Test Coverage**: Eight test modules covering all major components: authentication, CLI, settings, utils, applications, runs, and resources -- **Test Categories**: Sequential tests for order-dependent operations and long-running tests for performance validation -- **Fixture-based Setup**: Pytest fixtures for consistent test environments and mock configurations - ---- - -## 10. Implementation Details +## 9. Implementation Details -### 10.1 Key Algorithms +### 9.1 Key Algorithms and Business Logic - **PKCE Flow**: OAuth 2.0 Authorization Code flow with Proof Key for Code Exchange for enhanced security in public clients - **Token Caching**: File-based token persistence with expiration tracking and automatic cleanup - **Health Monitoring**: Multi-layer health checks including public endpoint availability and authenticated API access -### 10.2 State Management +### 9.2 State Management and Data Flow - **Configuration State**: Pydantic-based settings with environment variable override hierarchy - **Runtime State**: In-memory API client instances with lazy initialization - **Cache Management**: File-based token cache with automatic expiration and cleanup -### 10.3 Concurrency and Threading +### 9.3 Performance and Scalability Considerations - **Async Operations**: Synchronous design with thread-safe token operations - **Thread Safety**: File-based locking for token cache operations; atomic file writes for token storage --- - -## 11. Ketryx Field Mappings - -**For Ketryx Software Item Spec creation:** - -- **Description**: Platform authentication and API client management service for biomedical data analysis workflows -- **Introduced in version**: 1.0.0 -- **Parent software items**: Aignostics Python SDK Core -- **Fulfilled requirements**: REQ-PLATFORM-AUTH, REQ-TOKEN-MGMT, REQ-API-CLIENT -- **Software item type**: Platform Service -- **Safety risk class**: Class B - Handles authentication for medical data access but does not directly process patient data -- **Security risk class**: High - Manages authentication tokens and API access for biomedical platform -- **Inputs**: User credentials, environment configuration, API endpoints, file system access -- **Outputs**: Authenticated API clients, user information, health status, cached tokens -- **Used items**: OAuth 2.0 service, JWT validation, HTTP client libraries, file system operations -- **Rationale**: Provides secure, scalable authentication foundation required for biomedical data platform integration -- **Introduced risks**: RISK-AUTH-TOKEN (token security), RISK-NET-DEPEND (network dependency) -- **Context**: Platform Service for digital pathology and AI platform integration enabling secure access to biomedical analysis workflows - ---- diff --git a/specifications/SPEC_SYSTEM_SERVICE.md b/specifications/SPEC_SYSTEM_SERVICE.md index 7aa391fd..c138a5b2 100644 --- a/specifications/SPEC_SYSTEM_SERVICE.md +++ b/specifications/SPEC_SYSTEM_SERVICE.md @@ -198,28 +198,25 @@ graph LR - **Purpose**: Provides core system management, health monitoring, and configuration services - **Key Methods**: - - `health() -> Health`: Get aggregate system health including component status + - `health() -> Health`: Get aggregate system health including component status (instance method) - `health_static() -> Health`: Static method to get system health without instance - - `info(include_environ: bool, mask_secrets: bool) -> dict`: Static method to get comprehensive system information - - `is_token_valid(token: str) -> bool`: Validate authentication token + - `info(include_environ: bool = False, mask_secrets: bool = True) -> dict[str, Any]`: Static method to get comprehensive system information + - `is_token_valid(token: str) -> bool`: Validate authentication token (instance method) - `dotenv_set(key: str, value: str) -> None`: Static method to set environment variable in .env files - `dotenv_get(key: str) -> str | None`: Static method to get environment variable value - `dotenv_unset(key: str) -> int`: Static method to remove environment variable from .env files + - `remote_diagnostics_enable() -> None`: Static method to enable remote diagnostics + - `remote_diagnostics_disable() -> None`: Static method to disable remote diagnostics + - `http_proxy_enable(host: str, port: int, scheme: str, ssl_cert_file: str | None = None, no_ssl_verify: bool = False) -> None`: Static method to configure HTTP proxy + - `http_proxy_disable() -> None`: Static method to disable HTTP proxy + - `openapi_schema() -> JsonType`: Static method to get OpenAPI specification **Input/Output Contracts**: -- **Input Types**: Strings for tokens/keys/values, booleans for flags, timeout integers -- **Output Types**: Health objects, dictionaries for info, strings for configuration values +- **Input Types**: Strings for tokens/keys/values, booleans for flags, timeout integers, optional SSL certificate paths +- **Output Types**: Health objects, dictionaries for info, strings for configuration values, JSON for OpenAPI schema - **Error Conditions**: RuntimeError for network failures, ValueError for configuration errors, OpenAPISchemaError for schema issues -#### Static Methods Interface - -**Configuration Management**: - -- `remote_diagnostics_enable/disable() -> None`: Static methods to control remote diagnostics -- `http_proxy_enable/disable() -> None`: Static methods to configure HTTP proxy settings -- `openapi_schema() -> JsonType`: Static method to get OpenAPI specification - ### 4.2 CLI Interface **Command Structure:** @@ -230,28 +227,25 @@ uvx aignostics system [subcommand] [options] **Available Commands:** -| Command | Purpose | Input Requirements | Output Format | -| -------------------------- | ------------------------------ | ------------------------------ | --------------- | -| `health` | Display system health status | Optional output format | JSON/YAML | -| `info` | Show comprehensive system info | Optional environ/masking flags | JSON/YAML | -| `serve` | Start web GUI server | Host, port, browser options | Server startup | -| `openapi` | Display OpenAPI schema | API version, output format | JSON/YAML | -| `install` | Complete installation | None | Success message | -| `config get ` | Get configuration value | Configuration key name | Key value | -| `config set ` | Set configuration value | Key name and value | Success message | -| `config unset ` | Remove configuration value | Configuration key name | Success message | +| Command | Purpose | Input Requirements | Output Format | +| --------- | ------------------------------ | ------------------------------ | --------------- | +| `health` | Display system health status | Optional output format | JSON/YAML | +| `info` | Show comprehensive system info | Optional environ/masking flags | JSON/YAML | +| `serve` | Start web GUI server | Host, port, browser options | Server startup | +| `openapi` | Display OpenAPI schema | API version, output format | JSON/YAML | +| `install` | Complete installation | None | Success message | **Configuration Subcommands:** | Command | Purpose | Input Requirements | Output | | ----------------------------------- | -------------------------- | ----------------------- | --------------- | +| `config get ` | Get configuration value | Configuration key name | Key value | +| `config set ` | Set configuration value | Key name and value | Success message | +| `config unset ` | Remove configuration value | Configuration key name | Success message | | `config remote-diagnostics-enable` | Enable remote diagnostics | None | Success message | | `config remote-diagnostics-disable` | Disable remote diagnostics | None | Success message | | `config http-proxy-enable` | Configure HTTP proxy | Host, port, SSL options | Success message | | `config http-proxy-disable` | Disable HTTP proxy | None | Success message | -| `config remote-diagnostics-disable` | Disable remote diagnostics | None | Success message | -| `config http-proxy-enable` | Configure HTTP proxy | Host, port, SSL options | Success message | -| `config http-proxy-disable` | Disable HTTP proxy | None | Success message | ### 4.3 HTTP/Web Interface @@ -366,14 +360,7 @@ _Note: For exact version requirements, refer to `pyproject.toml` and dependency - **Token Security**: SecretStr usage for secure token storage and comparison - **Audit Logging**: Comprehensive logging of configuration changes and access -### 8.3 Secret Detection Algorithm - -The module implements sophisticated secret detection for environment variables: - -- **Word Boundary Matching**: Terms like "id" use regex word boundaries to avoid false positives -- **String Matching**: Unambiguous terms like "token", "key", "secret", "password" use substring matching -- **Case Insensitive**: All detection is case-insensitive for robustness -- **Real-world Patterns**: Handles common environment variable naming conventions +- **Token Security**: Automatic expiration, secure storage, validation on each use --- @@ -382,10 +369,20 @@ The module implements sophisticated secret detection for environment variables: ### 9.1 Key Algorithms and Business Logic - **Health Aggregation**: Automatic discovery and aggregation of BaseService implementations -- **System Information Gathering**: Comprehensive runtime, hardware, and process data collection -- **Secret Detection**: Dual-strategy pattern matching for environment variable classification +- **System Information Gathering**: Comprehensive runtime, hardware, and process data collection with configurable intervals +- **Secret Detection**: Dual-strategy pattern matching for environment variable classification using sophisticated algorithms: + - Word Boundary Matching: Terms like "id" use regex word boundaries to avoid false positives + - String Matching: Unambiguous terms like "token", "key", "secret", "password" use substring matching + - Case Insensitive: All detection is case-insensitive for robustness + - Real-world Patterns: Handles common environment variable naming conventions - **Configuration Management**: Atomic .env file operations with rollback capability +**Key Constants:** + +- `NETWORK_TIMEOUT = 5`: Network health check timeout in seconds +- `MEASURE_INTERVAL_SECONDS = 2`: CPU measurement interval for system info gathering +- `IPIFY_URL`: External service URL for network connectivity validation + ### 9.2 State Management and Data Flow - **State Type**: Mostly stateless with singleton service instances diff --git a/specifications/SPEC_WSI_SERVICE.md b/specifications/SPEC_WSI_SERVICE.md index fe9e9e15..265fc8af 100644 --- a/specifications/SPEC_WSI_SERVICE.md +++ b/specifications/SPEC_WSI_SERVICE.md @@ -38,10 +38,14 @@ The WSI Module shall: ### 1.4 Constraints and Limitations -- **File Format Support**: Limited to extensions defined in `WSI_SUPPORTED_FILE_EXTENSIONS = {".dcm", ".tiff", ".tif", ".svs"}` +- **File Format Support**: Limited to extensions defined in `WSI_SUPPORTED_FILE_EXTENSIONS = {".dcm", ".tiff", ".tif", ".svs"}` with format-specific capabilities: + - DICOM (.dcm): Complete metadata, pyramidal support, annotation support via PydicomHandler + - TIFF/BigTIFF (.tiff, .tif): Standard metadata, pyramidal support, limited annotations via OpenSlideHandler + - Aperio SVS (.svs): Vendor-specific metadata, pyramidal support, no annotation support via OpenSlideHandler - **OpenSlide Dependency**: Non-DICOM formats require OpenSlide library installation for processing - **Memory Constraints**: Large WSI files require careful memory management and streaming operations - **Platform Dependencies**: Requires platform-specific compilation of OpenSlide and image processing libraries +- **Coordinate Systems**: Supports DICOM image coordinates (pixel-based), QuPath coordinates (micron-based), and GeoJSON standard adaptation for pathology --- @@ -513,23 +517,3 @@ _Note: For exact version requirements, refer to `pyproject.toml` and dependency **Implementation**: See `src/aignostics/wsi/` for current implementation **Tests**: See `tests/aignostics/wsi/` for usage examples and verification **API Documentation**: Auto-generated from docstrings in service classes - ---- - -## Appendix A: File Format Support Matrix - -| Format | Extension | Handler | Pyramidal | Annotations | Metadata Level | -| ------ | ----------- | ---------------- | --------- | ----------- | --------------- | -| DICOM | .dcm | PydicomHandler | ✓ | ✓ | Complete | -| TIFF | .tiff, .tif | OpenSlideHandler | ✓ | Limited | Standard | -| SVS | .svs | OpenSlideHandler | ✓ | ✗ | Vendor-specific | - -## Appendix B: Supported Coordinate Systems - -- **DICOM Image Coordinates**: Pixel-based with origin at top-left -- **QuPath Coordinates**: Micron-based world coordinates -- **GeoJSON Standard**: Geographic coordinate system adaptation for pathology - ---- - -**Verification Status**: This specification has been verified against the actual source code implementation in the WSI module directory structure as of September 10, 2025.