Skip to content

Feat/generic event sourcing elasticsearch readmodel#13

Merged
eanzhao merged 48 commits intodevfrom
feat/generic-event-sourcing-elasticsearch-readmodel
Feb 25, 2026
Merged

Feat/generic event sourcing elasticsearch readmodel#13
eanzhao merged 48 commits intodevfrom
feat/generic-event-sourcing-elasticsearch-readmodel

Conversation

@loning
Copy link
Copy Markdown
Contributor

@loning loning commented Feb 23, 2026

This pull request makes significant improvements to the CI pipeline, adds new projection provider infrastructure, and updates documentation and solution structure to reflect recent architectural changes. The changes enhance build efficiency by splitting and optimizing jobs, add support for new read model providers (Elasticsearch and Neo4j), and clarify the event sourcing implementation and requirements.

CI/CD Pipeline Improvements:

  • Refactored .github/workflows/ci.yml to introduce job concurrency control, conditional job execution based on changed paths, and job splitting for faster and more targeted builds. Jobs are now grouped into fast gates, split-test guards (matrixed by solution filter), projection provider E2E tests, Kafka transport integration, and coverage quality checks. Each job only runs when relevant files are changed, improving CI efficiency. [1] [2] [3]

Projection Provider Infrastructure:

  • Added docker-compose.projection-providers.yml to provide local development services for Elasticsearch and Neo4j, supporting new projection provider implementations.
  • Updated the solution file aevatar.slnx to include new projects for Aevatar.CQRS.Projection.Runtime, Aevatar.CQRS.Projection.Providers.Elasticsearch, Aevatar.CQRS.Projection.Providers.InMemory, Aevatar.CQRS.Projection.Providers.Neo4j, and Aevatar.CQRS.Projection.StateMirror.

Documentation Updates:

  • Rewrote docs/EVENT_SOURCING.md to clarify the event sourcing baseline, lifecycle, DI/container conventions, snapshot semantics, and explicitly list prohibited patterns. The new document reflects the current and intended architecture, with precise references to code locations and configuration options.
  • Updated the description of ReadModel storage in README.md to clarify the default registration and provider mechanism for in-memory and persistent read models.

These changes collectively modernize the CI pipeline, enable new projection provider options, and provide clear, up-to-date documentation for developers.

- Introduced a comprehensive requirements document for Generic Event Sourcing and Index-Capable ReadModel, outlining the architecture, goals, and constraints.
- Defined key concepts such as State, ReadModel, and Provider capabilities, along with their interactions and expected behaviors.
- Specified functional and non-functional requirements, including event sourcing abstractions, state event models, and read model store semantics.
- Established guidelines for provider capabilities, metadata-driven indexing, and developer experience paths for integrating read models.
- Clarified the scope of the implementation, including in-scope and out-of-scope items, ensuring a clear understanding of the project's boundaries.
…Providers

- Introduced a new architecture for Provider-Based ReadModel, decoupling read model storage from specific business domains.
- Added Elasticsearch and InMemory providers for read model storage, supporting capabilities like indexing and schema validation.
- Implemented registration and selection mechanisms for providers, enhancing flexibility in read model management.
- Updated documentation to reflect the new architecture and provider capabilities, ensuring clarity for developers integrating read models.
- Refactored existing abstractions to support the new provider model, including capability validation and runtime options.
- Updated the Event Sourcing documentation to clarify the architecture and usage guidelines, ensuring consistency with the latest implementation.
- Introduced a new Neo4j provider for read models, enabling graph-based storage and retrieval capabilities.
- Added abstractions for provider registration, selection, and capability validation, enhancing the flexibility of the read model architecture.
- Implemented structured logging for write operations in the Neo4j provider, improving observability and debugging.
- Refactored existing abstractions to accommodate the new provider model and updated related documentation for clarity.
- Introduced `IStateEventApplier` interface and `StateEventApplierBase` class to facilitate modular state transitions in event sourcing.
- Added `StateTransitionMatcher` for deterministic event-to-state transitions, improving clarity and maintainability of state management.
- Updated `GAgentBase` to utilize new appliers for event handling, enhancing the flexibility of state transitions.
- Refactored existing event handling logic in `ProjectionOwnershipCoordinatorGAgent` and `WorkflowGAgent` to leverage the new state transition mechanisms.
- Enhanced documentation to reflect the new abstractions and usage patterns for developers.
- Added a CI guard to prevent direct mutations of `State.xxx` in any subclass of `GAgentBase<TState>`, enforcing the use of domain events and state transition mechanisms.
- Updated documentation to reflect the new restriction on state management practices.
- Enhanced existing architecture documentation to clarify the requirements for state transitions and event handling.
- Introduced a new `FileEventStore` for persistent event storage, allowing agents to maintain state across instances with optimistic concurrency checks.
- Added `FileEventStoreOptions` to configure the root directory for event storage.
- Updated `docker-compose.projection-providers.yml` to include Elasticsearch and Neo4j services, enhancing the development environment for testing projection providers.
- Implemented structured logging for write operations in both Elasticsearch and InMemory providers, improving observability.
- Enhanced tests for the new file-based event store and integration tests for Elasticsearch and Neo4j providers, ensuring robust functionality and reliability.
- Updated documentation to reflect the new event store capabilities and provider configurations, aiding developer understanding and usage.
- Introduced automatic snapshot functionality and event stream compaction in the event sourcing architecture, allowing for configurable snapshot intervals and retention policies.
- Updated `IEventStore` to support deletion of historical events based on snapshot versions, improving storage efficiency.
- Added `EventSourcingRuntimeOptions` to manage snapshot and compaction settings, enhancing flexibility for developers.
- Implemented in-memory and file-based snapshot stores to support the new snapshot capabilities.
- Updated documentation to reflect the new features and configuration options, aiding developer understanding and usage.
- Introduced `IEventStoreCompactionScheduler` to manage event compaction scheduling, allowing for deferred execution during idle actor lifecycles.
- Added `IActorDeactivationHook` and `IActorDeactivationHookDispatcher` to facilitate the execution of non-critical tasks post-actor deactivation, enhancing the runtime's lifecycle management.
- Implemented `EventStoreCompactionDeactivationHook` to trigger compaction on actor idle, improving event store efficiency.
- Updated `EventSourcingBehavior` to utilize the new compaction scheduler, ensuring historical event cleanup is handled asynchronously.
- Enhanced documentation to reflect the new compaction and deactivation hook features, aiding developer understanding and usage.
- Added tests for the new compaction scheduler and deactivation hooks to ensure reliability and correctness of the implementation.
…der Integration

- Updated the `WorkflowCapabilityServiceCollectionExtensions` to decouple ReadModel provider registration from the infrastructure layer, promoting a cleaner architecture.
- Introduced `WorkflowReadModelStartupValidationHostedService` to validate ReadModel provider capabilities during host startup, ensuring proper configuration and preventing runtime errors.
- Enhanced documentation to clarify the new provider registration process and startup validation features, aiding developer understanding.
- Added tests to verify the functionality of the new validation service and ensure correct behavior under various configurations.
- Updated CI guards to enforce architectural constraints regarding provider registrations, maintaining code quality and consistency.
- Introduced path filtering in the CI workflow to conditionally trigger jobs based on changes in specific directories or files, improving efficiency.
- Added new jobs for `split-test-guards`, `projection-provider-e2e`, and `kafka-transport-integration`, allowing for targeted testing and integration based on detected changes.
- Implemented a `coverage-quality` job to enforce coverage thresholds and ensure code quality during CI runs.
- Updated the CI configuration to include a comprehensive README, detailing the purpose and functionality of each script and job within the CI process, enhancing developer understanding and usability.
- Introduced a new GitHub Action, `Prepare Runner`, to streamline the setup of the .NET SDK, cache NuGet packages, and optionally install ripgrep.
- Updated multiple CI workflow jobs to utilize the new `Prepare Runner` action, enhancing maintainability and reducing redundancy in the setup steps.
- Added a new script, `restore_and_build.sh`, to centralize the restore and build process for the project, improving CI efficiency.
- Updated documentation to reflect the new action and script, providing clarity on their usage within the CI process.
- Updated `RoleGAgent` to utilize event sourcing for role configuration, ensuring state persistence through `HandleConfigureRoleAgent` method.
- Implemented state transition logic in `RoleGAgent` to manage role state changes effectively.
- Added `RoleGAgentReplayContractTests` to verify the correct behavior of role configuration persistence and replay functionality.
- Introduced `InMemoryEventStoreForTests` to facilitate testing of event sourcing mechanisms.
- Updated CI architecture guards to enforce replay contract testing requirements for stateful actors, ensuring comprehensive coverage and adherence to architectural standards.
- Introduced `GarnetEventStore` as a production-ready implementation of `IEventStore`, utilizing Redis for event persistence with optimistic concurrency support.
- Updated dependency injection to automatically configure `GarnetEventStore` when the persistence backend is set to Garnet, ensuring seamless integration with Orleans runtime.
- Enhanced documentation to include details on the new Garnet-backed event store and its configuration options.
- Added integration tests for `GarnetEventStore` to verify event appending, retrieval, and optimistic concurrency handling.
- Updated existing tests to ensure compatibility with the new persistence implementation, maintaining code quality and reliability.
- Added a new CI job `orleans-garnet-persistence-integration` to run the Orleans Garnet persistence smoke test, triggered by specific events such as pushes to `main` or `dev` branches and manual dispatch.
- Updated the README documentation to include details about the new job and its purpose.
- Refactored the `StatefulAgentSnapshot` test method to `StatefulAgentEventSourcedState` for clarity and improved state transition logic in the `RecordingGarnetStatefulAgent` class.
- Modified the `split-test-guards` job in the CI workflow to trigger on pushes to `main` and `dev` branches, as well as on scheduled events and manual dispatch.
- Updated the README documentation to clarify the new triggering conditions for the `split-test-guards` job, enhancing developer understanding of the CI process.
- Introduced `IEventSourcingBehaviorFactory` to streamline the creation of event sourcing behaviors, improving dependency management and reducing reliance on service locators.
- Updated `GAgentBase` to utilize the new factory for event sourcing behavior instantiation, enhancing code clarity and maintainability.
- Added `DefaultEventSourcingBehaviorFactory` to encapsulate event sourcing behavior creation logic, ensuring consistent configuration across agents.
- Implemented a new CI job `event-sourcing-regression` to run comprehensive tests for event sourcing, including core tests, Orleans integration, and architecture guards.
- Created `event_sourcing_regression.sh` script to automate the regression testing process, ensuring robust validation of event sourcing functionality.
- Updated CI documentation to reflect the new job and its purpose, enhancing developer understanding of the testing framework.
- Updated the CI workflow to include additional paths for projection-related files, ensuring comprehensive testing coverage.
- Introduced a new audit scorecard document for the Projection ReadModel, detailing the architecture evaluation and compliance metrics.
- Refactored the ProjectionReadModelStoreSelector to improve error handling with specific exceptions for provider selection issues.
- Implemented a WorkflowReadModelSelectionPlanner to centralize the logic for selecting read model providers, enhancing maintainability and clarity.
- Updated tests to validate the new selection planner and error handling mechanisms, ensuring robustness in provider selection processes.
- Introduced a new CI job `full-solution-regression` to run comprehensive tests on the full solution during pull requests, ensuring core code changes are validated.
- Added steps for preparing the runner, restoring dependencies, building the solution, and executing the regression tests with specific parameters to enhance testing coverage.
- Updated the CI workflow to improve overall testing efficiency and maintainability.
- Introduced a new document detailing the audit scorecard for the ReadModel Index as of 2026-02-24, outlining the audit scope, methods, and boundaries.
- Included a comprehensive evaluation of the architecture with a scoring system based on defined criteria, resulting in a perfect score of 100/100.
- Documented key evidence and validation results from various checks, ensuring transparency and accountability in the architecture assessment.
- Enhanced overall documentation for the ReadModel Index to support ongoing compliance and improvement efforts.
…ions

- Introduced a comprehensive blueprint for refactoring ReadModel graph relations, detailing the current capabilities, gaps, and restructuring goals.
- Added new abstractions for managing projection relation stores, including interfaces and classes for nodes, edges, and queries.
- Implemented support for relation capabilities in existing providers, enhancing the architecture to accommodate graph relations.
- Updated dependency injection configurations for InMemory and Neo4j providers to register relation stores, ensuring compatibility with the new abstractions.
- Enhanced validation mechanisms to include relation requirements and capabilities, improving overall robustness in provider selection and usage.
@eanzhao
Copy link
Copy Markdown
Contributor

eanzhao commented Feb 24, 2026

[Blocking] src/Aevatar.CQRS.Projection.Providers.Elasticsearch/DependencyInjection/ServiceCollectionExtensions.cs:26 + src/Aevatar.CQRS.Projection.Providers.Elasticsearch/Stores/ElasticsearchProjectionReadModelStore.cs:353:ProviderCapabilities 声明了 supportsAliases=true 和 supportsSchemaValidation=true,但实现中没有 alias/schema validation 对应逻辑;建议要么把两个 capability 改成 false,要么补齐真实实现,否则会误导 runtime 能力校验与 provider 选择。
[Blocking] src/Aevatar.CQRS.Projection.Providers.Elasticsearch/Stores/ElasticsearchProjectionReadModelStore.cs:76 + src/Aevatar.CQRS.Projection.Providers.Elasticsearch/Stores/ElasticsearchProjectionReadModelStore.cs:174:MutateAsync 是读改写且 Upsert 是普通 PUT,缺少 if_seq_no/if_primary_term 或 external version,重放/重试/并发时会有 stale write 覆盖;建议引入 OCC(至少二选一:ES seq_no/primary_term 或基于事件版本的 external version)。
[Blocking] src/workflow/extensions/Aevatar.Workflow.Extensions.Hosting/WorkflowProjectionProviderServiceCollectionExtensions.cs:26 + src/Aevatar.CQRS.Projection.Abstractions/Abstractions/ProjectionReadModelStoreSelector.cs:49:同一 read model 同时注册 3 个 provider,若配置未显式指定 provider 会触发“多 provider 未指定”异常;建议确保启动配置强制指定,或在组合层按环境只注册一个默认 provider。
[Major] src/Aevatar.CQRS.Projection.Providers.Elasticsearch/Stores/ElasticsearchProjectionReadModelStore.cs:105 + src/Aevatar.CQRS.Projection.Providers.Elasticsearch/Stores/ElasticsearchProjectionReadModelStore.cs:127:AutoCreateIndex=false 时索引缺失会被当成“无数据”(null/[]),容易掩盖配置错误;建议增加 strict 模式(索引不存在直接报错)或至少记录 warning/metric。
[Major] src/Aevatar.CQRS.Projection.Runtime/Runtime/ProjectionReadModelBindingResolver.cs:43:binding 解析优先 Type.Name 再 FullName,跨命名空间同名模型有误绑定风险;建议优先 FullName,或在发现重名 key 时 fail-fast。
[Minor] src/Aevatar.CQRS.Projection.Providers.Elasticsearch/Stores/ElasticsearchProjectionReadModelStore.cs:241:ListSortField 为空时 ListAsync 无排序,返回顺序不稳定;建议给默认排序字段(如 StartedAt)或在 README 明确“无序保证”。
[Minor] src/Aevatar.CQRS.Projection.Abstractions/Abstractions/ProjectionReadModelCapabilityValidator.cs:24:RequiredIndexKinds 用 Overlaps(任一匹配即通过),如果未来出现“必须同时满足多个 kind”的语义会放松校验;建议确认需求是否应为“全部包含”。

- Completed Phase 2 of the ReadModel graph relations refactor, implementing breaking changes to improve the architecture.
- Introduced `IProjectionStoreStartupValidator` for enhanced validation of read model and relation providers during startup.
- Split `IWorkflowExecutionProjectionPort` into `IWorkflowExecutionProjectionLifecyclePort` and `IWorkflowExecutionProjectionQueryPort`, clarifying responsibilities and improving maintainability.
- Updated dependency injection to register new lifecycle and query services, ensuring proper integration with the refactored architecture.
- Enhanced documentation to reflect changes in the projection architecture and the new validation mechanisms, providing clearer guidance for future development.
@loning loning requested a review from eanzhao February 24, 2026 07:57
- Updated Elasticsearch provider capabilities to align with actual implementations, removing unsupported alias/schema validation declarations.
- Introduced optimistic concurrency control (OCC) in `MutateAsync` to handle conflicts with retry logic.
- Implemented a new `MissingIndexBehavior` option to manage index absence scenarios, defaulting to fail-fast.
- Enforced binding key requirements to use `Type.FullName`, eliminating ambiguity in read model bindings.
- Adjusted provider registration to be conditional based on configuration, improving clarity and reducing potential errors.
- Enhanced documentation to reflect these architectural changes and provide clearer guidance for implementation.
- Enhanced the projection architecture by introducing `IProjectionStoreSelectionPlanner` and `IProjectionStoreSelectionRuntimeOptions` for improved provider selection and runtime options management.
- Removed the legacy `WorkflowReadModelSelectionPlanner` to streamline the selection process and reduce complexity.
- Updated the `WorkflowExecutionProjectionOptions` to eliminate direct provider configurations, centralizing them under the new projection runtime options.
- Enhanced the `WorkflowExecutionQueryApplicationService` to support new query options for actor relations, allowing for more flexible querying capabilities.
- Improved documentation to reflect architectural changes and provide clearer guidance on the new selection mechanisms and query options.
- Split the existing `Aevatar.CQRS.Projection.Abstractions` into two distinct projects: `Aevatar.CQRS.Projection.Core.Abstractions` and `Aevatar.CQRS.Projection.Stores.Abstractions` to clarify responsibilities and reduce complexity.
- Updated project references across the solution to align with the new structure, ensuring all dependencies point to the correct abstractions.
- Enhanced documentation to reflect the architectural changes and provide guidance on the new project organization.
- Removed the legacy `Aevatar.CQRS.Projection.Abstractions` project, streamlining the codebase and improving maintainability.
- Updated global using directives across multiple projects to reflect the new structure, replacing references to `Aevatar.CQRS.Projection.Abstractions` with `Aevatar.CQRS.Projection.Core.Abstractions` and `Aevatar.CQRS.Projection.Stores.Abstractions`.
- Removed the legacy `Aevatar.CQRS.Projection.Abstractions` references from the codebase, ensuring all components align with the refactored architecture.
- Enhanced overall code clarity and maintainability by streamlining the namespace usage in the projection-related components.
- Deleted outdated audit scorecard documents for Projection ReadModel, ReadModel Index, and Workflow subsolution to streamline the documentation and maintain focus on current architecture evaluations.
- Ensured that the removal aligns with the ongoing refactoring efforts and the transition to updated scorecard formats, enhancing overall documentation clarity and relevance.
- Introduced a detailed refactor plan for the Projection ReadModel, outlining core objectives, implementation strategies, and architectural changes aimed at enhancing clarity and maintainability.
- Added an audit scorecard for the Projection ReadModel, providing a comprehensive evaluation of the architecture with a scoring system and detailed validation results, ensuring transparency and accountability in the assessment process.
- Both documents align with ongoing refactoring efforts and aim to improve overall documentation quality and project organization.
…ilities

- Introduced new abstractions for document and graph read models, including `IDocumentReadModel`, `IGraphReadModel`, and their respective projection stores.
- Updated existing providers to support the new document and graph capabilities, including Elasticsearch and InMemory providers.
- Implemented a `ProjectionMaterializationRouter` for dual-write capabilities, allowing seamless integration between document and graph stores.
- Enhanced dependency injection configurations to register new services and ensure proper integration of the updated architecture.
- Improved documentation to reflect the architectural changes and provide clearer guidance on the new abstractions and provider capabilities.
…vider Updates

- Added `Aevatar.CQRS.Projection.Runtime.Abstractions` to support new runtime strategies and materialization contracts.
- Updated existing projection stores and providers to implement `IDocumentProjectionStore` and `IProjectionGraphStore` interfaces, enhancing document and graph capabilities.
- Refactored dependency injection configurations to register new services and ensure compatibility with the updated architecture.
- Improved documentation to reflect the architectural changes and provide clearer guidance on the new abstractions and provider capabilities.
…Clarity and Functionality

- Updated the Projection ReadModel documentation to reflect the latest architectural changes, including the removal of the capabilities model and the introduction of explicit provider selection.
- Refactored the Elasticsearch, InMemory, and Neo4j providers to eliminate deprecated capabilities and streamline provider registration.
- Introduced new startup validators for document and graph providers to ensure robust validation during initialization.
- Enhanced dependency injection configurations to support the updated architecture and improve service registration clarity.
- Improved overall documentation to provide clearer guidance on the new abstractions and provider functionalities.
…lection

- Updated the Projection ReadModel documentation to reflect the latest architectural changes, including the removal of the capabilities model and the introduction of explicit provider selection.
- Refactored dependency injection configurations to consolidate provider selection logic within factories, enhancing clarity and reducing complexity.
- Removed deprecated startup validators and provider registries, streamlining the initialization process for document and graph stores.
- Introduced new runtime options classes to replace previous selection options, ensuring a more straightforward configuration approach.
- Improved overall documentation to provide clearer guidance on the new abstractions and provider functionalities.
- Updated the Neo4j provider to focus solely on graph capabilities, removing document-related functionalities and associated configurations.
- Deleted deprecated classes and methods related to document handling in the Neo4j provider, streamlining the codebase.
- Enhanced the documentation for the Projection ReadModel to reflect the latest architectural changes, including the removal of the document provider from Neo4j and clarifications on provider selection.
- Improved error handling in the provider selection logic to prevent misconfigurations involving Neo4j as a document provider.
- Updated tests to ensure compliance with the new provider architecture and validate the changes made to the Neo4j integration.
- Completed the full refactor of the Projection ReadModel architecture, consolidating document and graph capabilities into distinct fan-out models.
- Removed the outdated audit scorecard for the Projection ReadModel and introduced a new scorecard reflecting the updated architecture and validation results.
- Enhanced documentation to clarify the new structure, including the removal of deprecated classes and the introduction of streamlined provider registration.
- Improved overall clarity and maintainability of the codebase by eliminating redundant abstractions and updating dependency injection configurations.
- Ensured that all changes align with the latest architectural objectives and enhance the overall functionality of the Projection ReadModel.
…agement Features

- Introduced a mechanism to explicitly specify a unique primary query provider for document and graph projections, enhancing clarity in multi-provider scenarios.
- Updated the `IProjectionStoreRegistration<TStore>` interface to include an `IsPrimaryQueryStore` property, ensuring only one primary provider is registered.
- Implemented new methods in `IProjectionGraphStore` and `InMemoryProjectionGraphStore` to support edge retrieval by owner, facilitating precise cleanup operations.
- Enhanced the `ProjectionGraphMaterializer` to manage edges based on ownership, improving the accuracy of graph updates and deletions.
- Updated documentation across various components to reflect these architectural changes and clarify the new functionalities.
- Updated the Projection ReadModel architecture to eliminate the need for an explicit primary query provider, simplifying the registration process for document and graph stores.
- Consolidated provider registration methods across Elasticsearch and InMemory providers, removing the `isPrimaryQueryStore` parameter to streamline configuration.
- Enhanced the `IProjectionGraphStore` interface with new methods for managing nodes and edges by owner, improving cleanup operations and data integrity.
- Updated the `ProjectionGraphMaterializer` to ensure accurate management of graph nodes and edges based on ownership, including the removal of stale nodes.
- Revised documentation to reflect these architectural changes and clarify the new provider registration semantics.
- Updated the Projection ReadModel architecture to utilize structured metadata objects for Elasticsearch, improving clarity and usability in index initialization.
- Enhanced the `DocumentIndexMetadata` to support structured mappings, settings, and aliases, replacing the previous JSON string representation.
- Refactored the `ElasticsearchProjectionReadModelStore` to consume the new structured metadata format, ensuring accurate index creation and configuration.
- Optimized the Neo4j subgraph query logic to reduce N+1 query risks by consolidating edge retrieval into a single Cypher query, improving performance in high-degree scenarios.
- Revised documentation to reflect these architectural changes and clarify the new metadata handling processes.
- Replaced `WorkflowProjectionLeaseManager` with `IProjectionOwnershipCoordinator` to streamline ownership management in the projection lifecycle.
- Updated related services and interfaces to utilize the new ownership coordinator, enhancing clarity and reducing complexity in the orchestration of projections.
- Removed deprecated interfaces for lease management and activation services, ensuring a cleaner codebase.
- Revised documentation to reflect the architectural changes and clarify the new ownership management processes.
- Introduced a new audit scorecard for the Projection storage architecture, detailing the objectives, scope, and evaluation criteria for assessing the relationship between DocumentStore and GraphStore.
- Defined a clear scoring methodology and outlined the current architectural conclusions, highlighting areas of redundancy and structural issues.
- Removed the outdated Projection Store / ReadModel scorecard to streamline documentation and focus on the new audit framework.
- Updated documentation to reflect the latest findings and recommendations for future architectural improvements and refactoring priorities.
- Added a new section detailing the target architecture diagram for the Projection storage, illustrating the relationship between ReadModel, UnifiedProjectionStoreDispatcher, and various StoreBindings.
- Clarified the semantics of the architecture, emphasizing the single-instance provider model for DocumentStore and GraphStore.
- Updated the redundancy issue list and refactored future refactoring priorities to align with the new architectural insights.
- Revised documentation to improve clarity and ensure consistency with the latest architectural framework.
- Replaced `IDocumentProjectionStore` with `IProjectionDocumentStore` across various components to unify the projection store interface.
- Updated the `ServiceCollectionExtensions` for both InMemory and Elasticsearch providers to reflect the new interface, enhancing clarity in service registration.
- Introduced `ElasticsearchProjectionDocumentStoreOptions` for better configuration management of Elasticsearch settings.
- Enhanced documentation to clarify the changes in projection store interfaces and their registration processes, ensuring consistency with the updated architecture.
- Deleted the obsolete Projection Store/ReadModel full refactor plan and audit scorecard documents, which were no longer aligned with the current architecture.
- Streamlined documentation to focus on the latest architectural insights and audit frameworks, ensuring clarity and relevance for future development.
- Updated related documentation to reflect the removal of deprecated content and maintain consistency with the current projection storage architecture.
- Introduced `IProjectionReadModel` interface to unify the ReadModel structure across different projection stores, ensuring consistency in implementation.
- Updated the `ServiceCollectionExtensions` for both InMemory and Elasticsearch providers to enforce the new `IProjectionReadModel` constraint, improving type safety and clarity in service registration.
- Added a new project reference for `Aevatar.CQRS.Projection.Stores.Abstractions` to facilitate integration with the updated ReadModel architecture.
- Revised documentation to reflect these changes and clarify the new ReadModel interface requirements, ensuring alignment with the latest architectural framework.
- Introduced `IStateMirrorReadModelProjector` interface to facilitate the projection of state to read models, enhancing the architecture's flexibility.
- Implemented `StateMirrorReadModelProjector` class to handle state projection and upsert operations, integrating with the existing projection store dispatcher.
- Updated `ServiceCollectionExtensions` to include new methods for registering state mirror projections and read model projectors, improving service configuration.
- Enhanced documentation to outline the new state mirror projection capabilities and provide usage examples, ensuring clarity for developers.
- Added a comprehensive audit scorecard for the state mirror projection architecture, detailing evaluation criteria and findings for future improvements.
…Classes

- Updated the `ElasticsearchProjectionDocumentStore` to improve code organization by splitting helper methods into dedicated support classes, enhancing maintainability and readability.
- Introduced `ElasticsearchProjectionDocumentStoreHttpSupport`, `ElasticsearchProjectionDocumentStoreMetadataSupport`, and `ElasticsearchProjectionDocumentStoreNamingSupport` to encapsulate specific functionalities related to HTTP operations, metadata normalization, and naming conventions.
- Removed the obsolete `ElasticsearchProjectionDocumentStore.Helpers.cs` file to streamline the codebase.
- Enhanced the `ServiceCollectionExtensions` for Elasticsearch to reflect the new class names, improving clarity in service registration.
- Updated documentation to reflect these structural changes and clarify the roles of the new support classes.
@eanzhao
Copy link
Copy Markdown
Contributor

eanzhao commented Feb 25, 2026

PR #13 代码审查报告(2026-02-25)


1. 结论

本次审查范围内的 F1-F4 问题均已落地修复,核心链路可用。

  • Blocking:0
  • Major:0
  • Medium:1(非阻断优化项)

合并建议:可合并。
剩余优化项:F2 outbox 的 completed 记录建议增加清理/归档策略,降低长期状态体积与扫描开销。


2. 问题与修复清单

ID 问题 状态 修复摘要
F1 ownership 可能长期占用 已修复 增加 lease TTL、续租与过期接管。
F2 双写失败补偿仅日志,可能读侧不一致 已修复 改为 actor 化 outbox + 异步 replay + backoff 重试。
F3 distributed 配置可能回退 InMemory 已修复 显式 durable provider + Document 侧门禁。
F4 启动校验未覆盖外部可达性 已修复 启动期改为真实 provider probe,并按环境分级处理失败。

3. 关键修复说明

F1:Ownership lease 过期与接管

  1. 协议增加 lease_ttl_ms
  2. Acquire 支持同 session renew 与过期 takeover。
  3. 新增 ProjectionOwnershipCoordinatorOptions(默认 TTL=30 分钟,可配置)。
  4. WorkflowExecutionProjectionOptions 新增 ProjectionOwnershipLeaseTtlMs 并注入 ownership coordinator。

关键代码:

  • src/Aevatar.CQRS.Projection.Core/projection_ownership_messages.proto
  • src/Aevatar.CQRS.Projection.Core/Orchestration/ProjectionOwnershipCoordinatorGAgent.cs
  • src/Aevatar.CQRS.Projection.Core/Orchestration/ProjectionOwnershipCoordinatorOptions.cs
  • src/Aevatar.CQRS.Projection.Core/Orchestration/ActorProjectionOwnershipCoordinator.cs
  • src/workflow/Aevatar.Workflow.Projection/Configuration/WorkflowExecutionProjectionOptions.cs

F2:双写失败补偿(actor 化 outbox)

  1. 补偿上下文扩展为 DispatchId / OccurredAtUtc / ReadModelType
  2. WorkflowProjectionDurableOutboxCompensator 在失败时入队补偿事件。
  3. ActorProjectionDispatchCompensationOutbox 通过 runtime 将事件投递到 outbox actor。
  4. WorkflowProjectionDispatchCompensationOutboxGAgent 维护持久状态并执行 replay。
  5. WorkflowProjectionDispatchCompensationReplayHostedService 周期触发 replay,失败 backoff,成功标记 completed。

关键代码:

  • src/Aevatar.CQRS.Projection.Core/projection_dispatch_compensation_messages.proto
  • src/workflow/Aevatar.Workflow.Projection/Orchestration/IProjectionDispatchCompensationOutbox.cs
  • src/workflow/Aevatar.Workflow.Projection/Orchestration/ActorProjectionDispatchCompensationOutbox.cs
  • src/workflow/Aevatar.Workflow.Projection/Orchestration/WorkflowProjectionDispatchCompensationOutboxGAgent.cs
  • src/workflow/Aevatar.Workflow.Projection/Orchestration/WorkflowProjectionDurableOutboxCompensator.cs
  • src/workflow/Aevatar.Workflow.Projection/Orchestration/WorkflowProjectionDispatchCompensationReplayHostedService.cs
  • src/workflow/Aevatar.Workflow.Projection/DependencyInjection/ServiceCollectionExtensions.cs

F3:distributed durable provider 与门禁

  1. appsettings.Distributed.json 显式启用 ES/Neo4j,关闭 InMemory。
  2. Document 侧新增 InMemory 门禁,与 Graph 策略对齐。

关键代码:

  • src/Aevatar.Mainnet.Host.Api/appsettings.Distributed.json
  • src/workflow/extensions/Aevatar.Workflow.Extensions.Hosting/WorkflowProjectionProviderServiceCollectionExtensions.cs

F4:启动探测升级

  1. 启动期改为真实 probe:
    • Document:ListAsync(take:1)
    • Graph:ListNodesByOwnerAsync(...)
  2. 失败策略:
    • Production:fail-fast
    • 非 Production:warning 并继续

关键代码:

  • src/workflow/Aevatar.Workflow.Projection/Orchestration/WorkflowReadModelStartupValidationHostedService.cs

4. 验证结果

  1. dotnet test test/Aevatar.Workflow.Host.Api.Tests/Aevatar.Workflow.Host.Api.Tests.csproj --nologo
    • 结果:Passed(160 passed / 0 failed)
  2. dotnet test test/Aevatar.CQRS.Projection.Core.Tests/Aevatar.CQRS.Projection.Core.Tests.csproj --nologo
    • 结果:Passed(55 passed / 1 skipped / 0 failed)
  3. bash tools/ci/architecture_guards.sh
    • 结果:Passed

5. 后续优化(非阻断)

  1. 为 F2 增加 outbox completed 记录清理/归档策略。
  2. 增加补偿运行指标:backlog、重试次数、重放延迟。
  3. 在 distributed smoke 中增加跨节点补偿收敛断言。

…docs

Co-authored-by: Cursor <cursoragent@cursor.com>
@eanzhao eanzhao force-pushed the feat/generic-event-sourcing-elasticsearch-readmodel branch from d07d220 to a545aa8 Compare February 25, 2026 07:23
…e handling in role agent configuration

- Introduced a new architecture audit report for PR #13, detailing the audit findings and scoring based on the defined criteria.
- Updated `ConfigureRoleAgentEvent` to use an optional field for temperature, allowing explicit zero values to be preserved.
- Modified `RoleGAgent` and `RoleGAgentFactory` to correctly handle the new temperature semantics, ensuring that explicit zero is retained during configuration.
- Enhanced projection ownership events to include an `occurred_at_utc` timestamp, improving event traceability and consistency.
- Added tests to verify the preservation of explicit zero temperature and the handling of occurred timestamps in projection events.
@loning
Copy link
Copy Markdown
Contributor Author

loning commented Feb 25, 2026

PR 标题建议
Refactor ownership lease replay semantics and role temperature presence (breaking)

PR 变更说明(可直接粘贴)

1. 背景与目标

本 PR 针对代码审查中的两个阻断问题做了彻底重构,并明确采用非兼容策略:

  1. 修复 Projection ownership lease 在重放时被“刷新为当前时间”的语义错误。
  2. 修复 Role YAML 配置中 temperature: 0 无法表达的语义丢失问题。

2. 核心改动

  1. Ownership lease 语义重构(P1)
  • 在 ownership 事件协议中新增 occurred_at_utc。
  • Acquire/Release 事件写入时显式携带事件发生时间。
  • 状态迁移只使用事件时间,不再使用 DateTime.UtcNow 更新 LastUpdatedAtUtc。
  • 去除兼容降级:重放时若事件缺少 occurred_at_utc,直接 fail-fast 抛错。
  1. Role temperature presence 语义重构(P2)
  • ConfigureRoleAgentEvent.temperature 从 double 改为 optional double。
  • YAML 应用配置时,只有在显式提供 temperature 才写入事件字段。
  • HandleConfigureRoleAgent 基于 HasTemperature 判断是否使用默认值,保留显式 0。

3. 破坏性变更(Breaking Changes)

  1. ProjectionOwnershipAcquireEvent/ProjectionOwnershipReleaseEvent 新增 occurred_at_utc 语义要求。
  2. 历史 ownership 事件若缺失 occurred_at_utc,激活重放将失败(不再做兼容回填)。
  3. ConfigureRoleAgentEvent.temperature 改为 optional,相关代码需按 presence 处理。

4. 测试与质量验证

已执行并通过:

  1. dotnet test aevatar.slnx --nologo
  2. dotnet test test/Aevatar.CQRS.Projection.Core.Tests/Aevatar.CQRS.Projection.Core.Tests.csproj --nologo --filter
    "FullyQualifiedNameProjectionOwnershipCoordinatorGAgentTests|FullyQualifiedNameActorProjectionOwnershipCoordinatorTests|
    FullyQualifiedName~ProjectionOwnershipProtoCoverageTests"
  3. dotnet test test/Aevatar.AI.Tests/Aevatar.AI.Tests.csproj --nologo --filter "FullyQualifiedNameRoleGAgentReplayContractTests|
    FullyQualifiedName
    AIHooksAndRoleFactoryCoverageTests|FullyQualifiedName~AIAbstractionsProtoCoverageTests"
  4. bash tools/ci/architecture_guards.sh
  5. bash tools/ci/test_stability_guards.sh

5. 新增/更新测试点

  1. ownership 重放保留时间戳并允许过期接管。
  2. ownership 缺失 occurred_at_utc 时重放 fail-fast。
  3. temperature: 0 显式值保留。
  4. 缺省 temperature 保持未设置(走 provider 默认)。

6. 影响范围

  1. CQRS Projection ownership 编排链路。
  2. AI Role YAML -> Event -> Runtime 配置链路。
  3. 对旧 ownership 历史数据存在不兼容影响(符合本次“无需兼容”要求)。

7. 文档更新

新增严格审计文档:
pr13-architecture-audit-strict-2026-02-25.md (/Users/cookie/aevatar/docs/audit-scorecard/pr13-architecture-audit-strict-2026-02-25.md)

关键变更文件

  • projection_ownership_messages.proto (/Users/cookie/aevatar/src/Aevatar.CQRS.Projection.Core/projection_ownership_messages.proto)
  • ProjectionOwnershipCoordinatorGAgent.cs (/Users/cookie/aevatar/src/Aevatar.CQRS.Projection.Core/Orchestration/
    ProjectionOwnershipCoordinatorGAgent.cs)
  • ActorProjectionOwnershipCoordinator.cs (/Users/cookie/aevatar/src/Aevatar.CQRS.Projection.Core/Orchestration/
    ActorProjectionOwnershipCoordinator.cs)
  • ai_messages.proto (/Users/cookie/aevatar/src/Aevatar.AI.Abstractions/ai_messages.proto)
  • RoleGAgentFactory.cs (/Users/cookie/aevatar/src/Aevatar.AI.Core/RoleGAgentFactory.cs)
  • RoleGAgent.cs (/Users/cookie/aevatar/src/Aevatar.AI.Core/RoleGAgent.cs)
  • ProjectionOwnershipAndSessionHubTests.cs (/Users/cookie/aevatar/test/Aevatar.CQRS.Projection.Core.Tests/
    ProjectionOwnershipAndSessionHubTests.cs)
  • ProjectionOwnershipProtoCoverageTests.cs (/Users/cookie/aevatar/test/Aevatar.CQRS.Projection.Core.Tests/
    ProjectionOwnershipProtoCoverageTests.cs)
  • RoleGAgentReplayContractTests.cs (/Users/cookie/aevatar/test/Aevatar.AI.Tests/RoleGAgentReplayContractTests.cs)

Copy link
Copy Markdown
Contributor

@eanzhao eanzhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eanzhao eanzhao merged commit be50ca1 into dev Feb 25, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants