A custom CSV reader implementation with direct file access
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
csv_position_reader
tests
.gitignore
LICENSE
README.md
setup.py

README.md

csv-position-reader

A custom CSV reader implementation with direct file access

The default builtin Python csv lib uses an 8KB read-ahead buffer on the file pointer, making fp.tell() yield inaccurate results. This library addresses that head on, explicitly passing back the file pointer position with each row, as well as allowing for direct seeking.

References:

Installation

pip install csv-position-reader

Usage

>>> import csv_position_reader

>>> with open('tests/data/basic.csv', 'r') as fp:
...     reader = csv_position_reader.DictReader(fp)
...     position, row = reader.next()
...     print "position: %s" % position
...     print "row: %s" % row
...     reader.seek(position)
...     position_new, row_new = reader.next()
...     assert position == position_new
...     assert row == row_new
... 
position: 26
row: {'city': 'Atlanta', 'favorite_color': 'black', 'name': 'Sam'}

Why? / Who Cares?

Because after poring through a CSV one time, you can now build a dictionary/cache of where each row lives for future O(1) access! You're now a stone's throw away from a CSV-driven database!