🚀 CSV Import Benchmark for .NET

Compare the performance of different asynchronous and concurrent approaches for importing 1,000,000 CSV records into MongoDB using .NET 10.

📖 Overview

This project compares several common data import strategies when processing a 1,000,000-row CSV file.

Each strategy performs exactly the same workflow:

CSV
 │
 ▼
Read
 │
 ▼
Parse
 │
 ▼
Validate
 │
 ▼
Insert MongoDB

The only difference is how concurrency is implemented.

The goal is to compare:

• 🚀 Throughput
• 💾 Memory Usage
• 🧹 GC Pressure
• 🧩 Code Complexity
• 🔧 Maintainability

---

✨ Benchmark Strategies

Strategy	Description
Loop	Sequential import
Task.WhenAll	Unlimited concurrent tasks
SemaphoreSlim	Limited concurrency
Channel	Producer-Consumer pipeline

---

📂 Project Structure

CsvImportBenchmark
│
├── Benchmark
│   └── CsvImportBenchmark.cs
│
├── Generator
│   └── CsvGenerator.cs
│
├── Importers
│   ├── LoopImporter.cs
│   ├── TaskWhenAllImporter.cs
│   ├── SemaphoreImporter.cs
│   └── ChannelImporter.cs
│
├── Models
│   └── User.cs
│
├── Mongo
│   ├── MongoContext.cs
│   └── UserRepository.cs
│
├── Parser
│   ├── CsvParser.cs
│   └── UserValidator.cs
│
└── Program.cs

---

⚙️ Tech Stack

• .NET 10
• MongoDB Driver
• CsvHelper
• Bogus
• BenchmarkDotNet
• System.Threading.Channels
• TPL Dataflow

---

📊 Benchmark

Run benchmarks:

dotnet run -c Release

Benchmark reports will be generated automatically:

BenchmarkDotNet.Artifacts/

└── results
    ├── *.md
    ├── *.html
    └── *.csv

---

📈 Example Result

BenchmarkDotNet v0.15.8, Windows 11 (10.0.26200.8655/25H2/2025Update/HudsonValley2)
13th Gen Intel Core i7-13700KF 3.40GHz, 1 CPU, 24 logical and 16 physical cores
.NET SDK 10.0.301
  [Host]     : .NET 10.0.9 (10.0.9, 10.0.926.27113), X64 RyuJIT x86-64-v3
  Job-CNUJVU : .NET 10.0.9 (10.0.9, 10.0.926.27113), X64 RyuJIT x86-64-v3

InvocationCount=1  UnrollFactor=1

Method	Mean	Error	StdDev	Median	Ratio	RatioSD	Gen0	Gen1	Gen2	Allocated	Alloc Ratio
Loop	84.01 s	1.414 s	1.181 s	84.25 s	1.00	0.02	1319000.0000	64000.0000	-	19.27 GB	1.00
TaskWhenAll	NA	NA	NA	NA	?	?	NA	NA	NA	NA	?
Semaphore	30.20 s	3.886 s	11.457 s	22.76 s	0.36	0.14	1333000.0000	403000.0000	3000.0000	19.35 GB	1.00
Channel	14.42 s	0.305 s	0.876 s	14.26 s	0.17	0.01	1332000.0000	191000.0000	190000.0000	19.24 GB	1.00

Benchmarks with issues: CsvImportBenchmark.TaskWhenAll: Job-CNUJVU(InvocationCount=1, UnrollFactor=1)

Actual benchmark results depend on CPU, disk, MongoDB configuration, and hardware.

---

📝 Blog Series

This repository accompanies the following article series:

1. Introduction
2. Generate 1,000,000 CSV Records
3. Loop Import
4. Task.WhenAll
5. SemaphoreSlim
6. Channel
7. TPL Dataflow
8. Benchmark Comparison

---

🚀 Future Improvements

• Parallel.ForEachAsync
• InsertManyAsync
• BulkWriteAsync
• Batch Import
• PipeReader
• Span
• MemoryPool
• Zero Allocation CSV Parsing

---

🤝 Contributing

Pull Requests and Issues are welcome.

If you have a better implementation or optimization idea, feel free to open an Issue or submit a PR.

---

⭐ Support

If this project helps you, please consider giving it a Star ⭐.