Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Oct 26, 2025

The source generator creates static ResourcePackageReader instances shared across threads. Without synchronization, concurrent reads corrupt data as FileStream Seek/Read operations interleave.

Changes

  • ResourcePackageReader: Optimized for true concurrent reads using FileShare.Read

    • Store file path instead of persistent FileStream
    • ReadResourceAsync(), ReadResource(): Create independent FileStream per operation with FileShare.Read
    • GetStream(): Returns ResourceSubStream for true streaming without loading data into memory
    • Removed: IDisposable interface (no longer holds persistent resources)
    • Removed: Lock-based synchronization (not needed with FileShare.Read)
  • ResourceSubStream: New internal class for streaming large resources

    • Lazy-loads FileStream on first read
    • Supports Seek and Read operations for streaming access
    • Implements IDisposable for proper cleanup
    • Thread-safe (each instance has its own FileStream)
  • Thread-safety tests: 7 tests validating concurrent access patterns (async/sync reads, streams, mixed operations)

Performance Benefits

  • ✅ True parallel reads - multiple threads can read simultaneously without blocking
  • ✅ No lock contention overhead
  • ✅ Memory efficient - GetStream() supports streaming large resources without loading into memory
  • ✅ Simpler, cleaner code (no IDisposable on ResourcePackageReader itself)
  • FileStream creation overhead per operation is minimal and handled efficiently by the OS

Example

Generated static accessor now safe for concurrent use:

// Multiple threads can safely call this simultaneously
Parallel.For(0, 100, i => {
    var data = MyAssemblyResourceAccess.ReadMyResource();
    // No data corruption, true parallel execution
});

// Stream large resources without memory overhead
using (var stream = MyAssemblyResourceAccess.GetMyLargeResourceStream())
{
    // Process stream incrementally without loading all data into memory
    // Stream is IDisposable and properly releases FileStream
}

Each read operation creates its own FileStream with FileShare.Read, allowing the OS to handle concurrent file access efficiently without application-level locking.

Original prompt

This section details on the original issue you should resolve

<issue_title>确保 ResourcePackageReader 是线程安全的</issue_title>
<issue_description>在生成了静态快速访问方法后,ResourcePackageReader 的线程安全就比较重要了。如何让 ResourcePackageReader 线程安全呢?</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits October 26, 2025 03:20
Co-authored-by: Soar360 <15421284+Soar360@users.noreply.github.com>
Co-authored-by: Soar360 <15421284+Soar360@users.noreply.github.com>
Copilot AI changed the title [WIP] Ensure ResourcePackageReader is thread-safe Make ResourcePackageReader thread-safe for concurrent access Oct 26, 2025
Copilot AI requested a review from Soar360 October 26, 2025 03:27
Copilot finished work on behalf of Soar360 October 26, 2025 03:27
@Soar360
Copy link
Contributor

Soar360 commented Oct 26, 2025

我有几个小问题

  1. 我们真的有必要一直持有 FileStream 对象吗?还是说可以用到时再创建?
  2. 我记得 FileStream 可以共享读,而不是非要加锁。这里是不是可以优化一下?
  3. 如果改为共享读,性能是会升高还是降低?

@copilot

Co-authored-by: Soar360 <15421284+Soar360@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 26, 2025

我有几个小问题

  1. 我们真的有必要一直持有 FileStream 对象吗?还是说可以用到时再创建?
    ...

优秀的观察!你完全正确。我已经按照你的建议优化了实现:

回答你的问题:

  1. 是否必须持有 FileStream? 不必要。现在改为存储文件路径,需要时创建 FileStream。

  2. FileStream 可以共享读? 完全正确!使用 FileShare.Read 允许多个 FileStream 同时读取同一文件,不需要锁。

  3. 性能影响? 性能会提升

    • ✅ 真正的并行读取(无锁竞争)
    • ✅ 多线程可同时读取不同资源
    • ✅ 代码更简单(删除了 SubStream 类和锁逻辑)
    • 开销:每次读取创建 FileStream(OS 层面高效处理)

变更(commit 62be3a1):

  • 存储 _filePath 而非 _fileStream
  • 每个读操作创建独立的 FileStream(带 FileShare.Read)
  • 完全移除锁机制
  • 简化 GetStream() 返回 MemoryStream
  • 删除 SubStream 类(121 行代码减少)

所有 36 个测试通过。这是更优的线程安全实现。

Copilot AI changed the title Make ResourcePackageReader thread-safe for concurrent access Make ResourcePackageReader thread-safe using FileShare.Read Oct 26, 2025
Copilot finished work on behalf of Soar360 October 26, 2025 03:39
@Soar360
Copy link
Contributor

Soar360 commented Oct 26, 2025

@copilot 你做的很好,还需要解决以下问题:

1、从现在看,ResourcePackageReader 已经没有必要实现 IDispose 接口了。
2、暴露 GetStream 方法的本意就是为了在遇到比较大的资源时进行流式处理,不把全部数据读入内存,以降低内存压力。所以 GetStream 方法的实现要重新设计。

Co-authored-by: Soar360 <15421284+Soar360@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 26, 2025

@copilot 你做的很好,还需要解决以下问题:

1、从现在看,ResourcePackageReader 已经没有必要实现 IDispose 接口了。
...

完成!两个问题都已解决(commit f7d1483):

1. 移除 IDisposable 接口

  • 删除 IDisposable 实现(不再需要,无持久化资源)
  • 移除所有 using 语句和 Dispose 相关测试

2. 重新设计 GetStream() 支持真正的流式处理

  • 创建 ResourceSubStream
  • 延迟加载 FileStream(首次读取时创建)
  • 支持大文件流式访问,不占用内存
  • Stream 本身实现 IDisposable,正确释放资源
  • 线程安全(每个 Stream 有独立的 FileStream)

优势:

  • ✅ 处理大资源时内存占用低
  • ✅ 支持 Seek/Read 流式操作
  • ✅ 返回的 Stream 可用 using 自动释放
  • ✅ 完全线程安全

所有 32 个测试通过。

Copilot finished work on behalf of Soar360 October 26, 2025 03:51
@Soar360 Soar360 marked this pull request as ready for review October 26, 2025 03:55
@Soar360 Soar360 merged commit 83f4186 into main Oct 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

确保 ResourcePackageReader 是线程安全的

2 participants