Skip to content

Convert VerifyFileHash and FromCompletedFile to async#23

Merged
yordis merged 1 commit intomasterfrom
async
Jul 15, 2025
Merged

Convert VerifyFileHash and FromCompletedFile to async#23
yordis merged 1 commit intomasterfrom
async

Conversation

@j-fuxa
Copy link
Contributor

@j-fuxa j-fuxa commented Jul 14, 2025

Convert VerifyFileHash and FromCompletedFile to async

@coderabbitai
Copy link

coderabbitai bot commented Jul 14, 2025

Walkthrough

This change refactors several core and test components to make chunk file initialization, verification, and related operations fully asynchronous with cancellation support. It updates method signatures, internal calls, and control flow to use async/await patterns, introduces cancellation tokens, and improves error handling and validation logic across chunk management and redaction services.

Changes

File(s) Change Summary
src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunk.cs,
src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunkReadSide.cs
Refactored TFChunk to make FromCompletedFile, InitCompleted, and VerifyFileHash methods asynchronous with cancellation support; changed field visibility; updated midpoints depth usage.
src/EventStore.Core/TransactionLog/Chunks/TFChunkDb.cs,
src/EventStore.Core/TransactionLog/Chunks/TFChunkManager.cs
Updated chunk loading and verification to use async/await with cancellation tokens; replaced lambdas with method group references; background verification now asynchronous.
src/EventStore.Core/Services/RedactionService.cs Split into abstract and generic classes; refactored to use async handlers and improved validation/error reporting for chunk switching and event position retrieval; added cancellation support and result type pattern for validation.
src/EventStore.Core.Tests/TransactionLog/when_opening_existing_tfchunk.cs,
src/EventStore.Core.Tests/TransactionLog/when_reading_from_a_cached_tfchunk.cs,
src/EventStore.Core.Tests/TransactionLog/when_uncaching_a_tfchunk.cs
Updated test setup to await asynchronous chunk initialization methods.
src/EventStore.Core.Tests/TransactionLog/when_opening_tfchunk_from_non_existing_file.cs Renamed test class to PascalCase; updated test to use async assertion and await chunk initialization.
src/EventStore.Core.Tests/Transforms/TransformTests.cs Updated checksum verification method to be asynchronous; propagated async calls and cancellation token usage.

Sequence Diagram(s)

sequenceDiagram
    participant Test as Test/Service
    participant TFChunk as TFChunk
    participant File as File System

    Test->>+TFChunk: await FromCompletedFile(filename, ..., token)
    TFChunk->>+File: Open file async
    TFChunk->>TFChunk: await InitCompleted(..., token)
    TFChunk->>+TFChunk: await VerifyFileHash(token)
    TFChunk->>-File: Read and verify hash
    TFChunk-->>-Test: TFChunk instance (when complete)
Loading
sequenceDiagram
    participant RedactionService
    participant TFChunkDb
    participant TFChunk

    RedactionService->>+TFChunkDb: Validate chunk switch (async, token)
    TFChunkDb->>+TFChunk: await FromCompletedFile(newChunkFile, ..., token)
    TFChunk->>+File: Open and verify file hash async
    TFChunk-->>-TFChunkDb: Validated chunk or error
    TFChunkDb-->>-RedactionService: Result (success or failure)
    RedactionService-->>Caller: Reply with result
Loading

Possibly related PRs

  • TrogonStack/TrogonEventStore#15: Refactors TFChunk read methods to be asynchronous with cancellation tokens, similar to this PR's async refactoring of chunk initialization and verification.

Poem

In a warren of code, where the chunks hop along,
Async now rules, and the waits aren’t as long.
With tokens for stopping and hashes to check,
The rabbits of code keep the bugs in check.
So here’s to the chunks, now swift and polite—
They load up their data and do it just right!
🐇✨


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ab6bde7 and f58c08d.

📒 Files selected for processing (10)
  • src/EventStore.Core.Tests/TransactionLog/when_opening_existing_tfchunk.cs (1 hunks)
  • src/EventStore.Core.Tests/TransactionLog/when_opening_tfchunk_from_non_existing_file.cs (1 hunks)
  • src/EventStore.Core.Tests/TransactionLog/when_reading_from_a_cached_tfchunk.cs (1 hunks)
  • src/EventStore.Core.Tests/TransactionLog/when_uncaching_a_tfchunk.cs (1 hunks)
  • src/EventStore.Core.Tests/Transforms/TransformTests.cs (4 hunks)
  • src/EventStore.Core/Services/RedactionService.cs (2 hunks)
  • src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunk.cs (8 hunks)
  • src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunkReadSide.cs (1 hunks)
  • src/EventStore.Core/TransactionLog/Chunks/TFChunkDb.cs (6 hunks)
  • src/EventStore.Core/TransactionLog/Chunks/TFChunkManager.cs (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • src/EventStore.Core.Tests/TransactionLog/when_reading_from_a_cached_tfchunk.cs
🚧 Files skipped from review as they are similar to previous changes (9)
  • src/EventStore.Core.Tests/TransactionLog/when_opening_existing_tfchunk.cs
  • src/EventStore.Core/TransactionLog/Chunks/TFChunkManager.cs
  • src/EventStore.Core.Tests/TransactionLog/when_uncaching_a_tfchunk.cs
  • src/EventStore.Core.Tests/Transforms/TransformTests.cs
  • src/EventStore.Core/TransactionLog/Chunks/TFChunkDb.cs
  • src/EventStore.Core.Tests/TransactionLog/when_opening_tfchunk_from_non_existing_file.cs
  • src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunkReadSide.cs
  • src/EventStore.Core/Services/RedactionService.cs
  • src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunk.cs
✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@j-fuxa j-fuxa force-pushed the async branch 4 times, most recently from 137a039 to 8bbcd5c Compare July 15, 2025 14:21
@j-fuxa j-fuxa marked this pull request as ready for review July 15, 2025 14:28
@j-fuxa j-fuxa requested a review from yordis July 15, 2025 14:28
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunk.cs (1)

304-304: Remove unnecessary empty line.

-		Func<TransformType, IChunkTransformFactory> getTransformFactory, CancellationToken token)
-	{
-
-		var fileInfo = new FileInfo(_filename);
+		Func<TransformType, IChunkTransformFactory> getTransformFactory, CancellationToken token)
+	{
+		var fileInfo = new FileInfo(_filename);
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7b3f176 and 8bbcd5c.

📒 Files selected for processing (10)
  • src/EventStore.Core.Tests/TransactionLog/when_opening_existing_tfchunk.cs (1 hunks)
  • src/EventStore.Core.Tests/TransactionLog/when_opening_tfchunk_from_non_existing_file.cs (1 hunks)
  • src/EventStore.Core.Tests/TransactionLog/when_reading_from_a_cached_tfchunk.cs (1 hunks)
  • src/EventStore.Core.Tests/TransactionLog/when_uncaching_a_tfchunk.cs (1 hunks)
  • src/EventStore.Core.Tests/Transforms/TransformTests.cs (4 hunks)
  • src/EventStore.Core/Services/RedactionService.cs (2 hunks)
  • src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunk.cs (8 hunks)
  • src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunkReadSide.cs (1 hunks)
  • src/EventStore.Core/TransactionLog/Chunks/TFChunkDb.cs (6 hunks)
  • src/EventStore.Core/TransactionLog/Chunks/TFChunkManager.cs (1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (3)
src/EventStore.Core.Tests/TransactionLog/when_uncaching_a_tfchunk.cs (1)
src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunk.cs (3)
  • TFChunk (27-1618)
  • TFChunk (174-192)
  • TFChunk (194-197)
src/EventStore.Core/TransactionLog/Chunks/TFChunkDb.cs (2)
src/EventStore.Core/TransactionLog/Chunks/TFChunkManager.cs (2)
  • TFChunk (436-448)
  • TFChunk (450-461)
src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunk.cs (3)
  • TFChunk (27-1618)
  • TFChunk (174-192)
  • TFChunk (194-197)
src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunk.cs (5)
src/EventStore.Core/TransactionLog/Chunks/TFChunkDb.cs (3)
  • ValueTask (59-264)
  • ValueTask (385-385)
  • ValueTask (387-412)
src/EventStore.Core/TransactionLog/Chunks/TFChunkManager.cs (10)
  • ValueTask (49-63)
  • ValueTask (79-126)
  • ValueTask (128-146)
  • ValueTask (148-181)
  • ValueTask (183-223)
  • ValueTask (238-257)
  • ValueTask (259-341)
  • ValueTask (463-482)
  • TFChunk (436-448)
  • TFChunk (450-461)
src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunkReadSide.cs (1)
  • TFChunk (12-719)
src/EventStore.Core/TransactionLog/Chunks/TFConsts.cs (1)
  • TFConsts (4-17)
src/EventStore.Core/Util/MD5Hash.cs (1)
  • MD5Hash (8-43)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: Build / noble
  • GitHub Check: Build / bookworm-slim
  • GitHub Check: Build / ubuntu-24.04
  • GitHub Check: Build / ubuntu-24.04-arm
🔇 Additional comments (15)
src/EventStore.Core.Tests/TransactionLog/when_uncaching_a_tfchunk.cs (1)

38-40: LGTM: Correctly awaiting the async FromCompletedFile method.

The change properly awaits the asynchronous TFChunk.FromCompletedFile method, which now returns ValueTask<TFChunk> instead of TFChunk. This aligns with the broader async refactoring of chunk initialization methods.

src/EventStore.Core.Tests/TransactionLog/when_opening_existing_tfchunk.cs (1)

23-25: LGTM: Correctly awaiting the async FromCompletedFile method.

The change properly awaits the asynchronous TFChunk.FromCompletedFile method, maintaining consistency with the async refactoring across the codebase.

src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunkReadSide.cs (1)

189-189: LGTM: Correctly using the encapsulated private field.

The change from Chunk.MidpointsDepth to Chunk._midpointsDepth reflects the encapsulation improvement where the field was made private and renamed to follow C# naming conventions for private fields.

src/EventStore.Core.Tests/TransactionLog/when_reading_from_a_cached_tfchunk.cs (1)

38-40: LGTM: Correctly awaiting the async FromCompletedFile method.

The change properly awaits the asynchronous TFChunk.FromCompletedFile method, ensuring consistent async handling across the test suite.

src/EventStore.Core.Tests/TransactionLog/when_opening_tfchunk_from_non_existing_file.cs (2)

10-10: LGTM: Improved class naming to follow C# conventions.

The class name change from lowercase to PascalCase follows proper C# naming conventions for class names.


15-17: LGTM: Correctly updated assertion for async method.

The change from Assert.Throws to Assert.ThrowsAsync is necessary because TFChunk.FromCompletedFile is now asynchronous. The async lambda properly awaits the method call to test that the expected exception is thrown.

src/EventStore.Core/TransactionLog/Chunks/TFChunkManager.cs (1)

305-307: LGTM! Async conversion implemented correctly.

The synchronous call to TFChunk.FromCompletedFile has been properly converted to async with await, and the cancellation token is correctly passed through.

src/EventStore.Core.Tests/Transforms/TransformTests.cs (1)

5-5: LGTM! Test methods properly converted to async.

The test has been correctly updated to support the async chunk verification:

  • Added necessary using directive for threading
  • Properly converted VerifyChecksums to async ValueTask with cancellation support
  • Correctly awaits the async VerifyFileHash method

Also applies to: 56-56, 67-67, 85-85

src/EventStore.Core/TransactionLog/Chunks/TFChunkDb.cs (2)

94-100: LGTM! Async chunk loading implemented correctly.

The synchronous chunk loading calls have been properly converted to async:

  • All FromCompletedFile and FromOngoingFile calls correctly use await
  • Cancellation tokens are properly propagated
  • Good simplification using method group reference TransformManager.GetFactoryForExistingChunk instead of lambda

Also applies to: 108-109, 117-123, 170-176, 206-207


231-260: LGTM! Background verification properly converted to async.

The background chunk verification has been correctly updated:

  • Uses async delegate with proper cancellation token propagation
  • Correctly awaits the async VerifyFileHash method
  • Maintains the same error handling logic
src/EventStore.Core/TransactionLog/Chunks/TFChunk/TFChunk.cs (3)

90-90: Good encapsulation improvement.

Changing MidpointsDepth from public to private field _midpointsDepth improves encapsulation and follows proper naming conventions for private fields.

Also applies to: 185-185


199-217: LGTM! Async initialization methods implemented correctly.

The factory methods and initialization methods have been properly converted to async:

  • FromCompletedFile correctly returns ValueTask<TFChunk>
  • InitCompleted properly handles async operations with cancellation
  • All async calls use await appropriately
  • Cancellation tokens are propagated correctly

Also applies to: 301-359


606-632: Excellent async conversion and optimization.

The VerifyFileHash method has been properly converted to async with two improvements:

  1. Added cancellation support with proper token checking
  2. Replaced manual byte array comparison with more efficient MemoryExtensions.SequenceEqual
src/EventStore.Core/Services/RedactionService.cs (2)

19-56: Good architectural refactoring.

The refactoring to split the service into an abstract base class and a generic derived class improves the design and allows for better type safety with the generic TStreamId parameter.


206-323: LGTM! Robust async validation implementation.

The IsValidSwitchChunkRequest method has been properly converted to async with excellent improvements:

  • Uses Result<TFChunk, SwitchChunkResult> for proper error handling
  • Comprehensive validation checks for file safety and chunk integrity
  • Proper async file stream usage with await using
  • Correctly propagates cancellation token to TFChunk.FromCompletedFile

Signed-off-by: JOSE FUXA <9057699+j-fuxa@users.noreply.github.com>
@j-fuxa
Copy link
Contributor Author

j-fuxa commented Jul 15, 2025

@coderabbitai review

@coderabbitai
Copy link

coderabbitai bot commented Jul 15, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@yordis yordis added this pull request to the merge queue Jul 15, 2025
Merged via the queue into master with commit 696002c Jul 15, 2025
9 checks passed
@yordis yordis deleted the async branch July 15, 2025 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants