Fork of Sebastien Lorion's Fast CSV Reader
C# ASP
Switch branches/tags
Nothing to show
Pull request Compare This branch is 3 commits ahead of Sjord:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
Assemblies/NUnit-2.5.10.11092/net-2.0
CsvReaderBenchmarks
CsvReaderDemo
CsvReaderDemoWeb
LumenWorks.Framework.IO
LumenWorks.Framework.Tests.Unit
Sjoerder.CsvWrapper.Tests.Unit
Sjoerder.CsvWrapper
.gitignore
CsvReader.sln
README.markdown
license.txt

README.markdown

A Fast CSV Reader

By Sebastien Lorion

#Preface This is a fork of Sebastien Lorion original "A-Fast CSV Reader" article on codeproject.

Sjoerd Langkemper already put this code on to Github and extended it, adding an extra wrapper and more tests.

Also consider looking at NLight - in particular NLight.IO - a more recent CSV parser by Sebastien Lorion. #Introduction From CodeProject From Original article

One would imagine that parsing CSV files is a straightforward and boring task. I was thinking that too, until I had to parse several CSV files of a couple GB each. After trying to use the OLEDB JET driver and various Regular Expressions, I still ran into serious performance problems. At this point, I decided I would try the custom class option. I scoured the net for existing code, but finding a correct, fast, and efficient CSV parser and reader is not so simple, whatever platform/language you fancy.

I say correct in the sense that many implementations merely use some splitting method like String.Split(). This will, obviously, not handle field values with commas. Better implementations may care about escaped quotes, trimming spaces before and after fields, etc., but none I found were doing it all, and more importantly, in a fast and efficient manner.

And, this led to the CSV reader class I present in this article. Its design is based on the System.IO.StreamReader class, and so is a non-cached, forward-only reader (similar to what is sometimes called a fire-hose cursor).

Benchmarking it against both OLEDB and regex methods, it performs about 15 times faster, and yet its memory usage is very low.

To give more down-to-earth numbers, with a 45 MB CSV file containing 145 fields and 50,000 records, the reader was processing about 30 MB/sec. So all in all, it took 1.5 seconds! The machine specs were P4 3.0 GHz, 1024 MB.

Read the rest of the article here

#Licence Original Copyright (c) 2005-2011 Sébastien Lorion

Copyright (c) 2011 Sjoerd Langkemper

Copyright (c) 2012 Paul Evans and contributors

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.